[==========] Running 26 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 8 tests from TxnWriteOpsITest
[ RUN ] TxnWriteOpsITest.CommitTimestampPropagation
2025-01-11T09:01:10Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-01-11T09:01:10Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20250111 09:01:10.643951 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.61.62:43521
--webserver_interface=127.14.61.62
--webserver_port=0
--builtin_ntp_servers=127.14.61.20:36263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.61.62:43521
--txn_manager_enabled=true
--txn_manager_lazily_initialized=false with env {}
W20250111 09:01:10.734020 14588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:01:10.734215 14588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:01:10.734285 14588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:01:10.736891 14588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250111 09:01:10.736943 14588 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20250111 09:01:10.736970 14588 flags.cc:425] Enabled experimental flag: --txn_manager_lazily_initialized=false
W20250111 09:01:10.737012 14588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:01:10.737053 14588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250111 09:01:10.737093 14588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250111 09:01:10.740108 14588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:36263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.61.62:43521
--txn_manager_enabled=true
--txn_manager_lazily_initialized=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.61.62:43521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.61.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.14588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:01:10.740890 14588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:01:10.741883 14588 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:01:10.747886 14593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:10.748090 14594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:10.748769 14596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:01:10.749553 14588 server_base.cc:1034] running on GCE node
I20250111 09:01:10.749943 14588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:01:10.750744 14588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:01:10.751888 14588 hybrid_clock.cc:648] HybridClock initialized: now 1736586070751876 us; error 28 us; skew 500 ppm
I20250111 09:01:10.753571 14588 webserver.cc:458] Webserver started at http://127.14.61.62:45367/ using document root <none> and password file <none>
I20250111 09:01:10.754055 14588 fs_manager.cc:362] Metadata directory not provided
I20250111 09:01:10.754113 14588 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:01:10.754263 14588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:01:10.755642 14588 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/master-0/data/instance:
uuid: "28fc30b9e1394ddbbdddfb34efd25b49"
format_stamp: "Formatted at 2025-01-11 09:01:10 on dist-test-slave-nvbt"
I20250111 09:01:10.756026 14588 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/master-0/wal/instance:
uuid: "28fc30b9e1394ddbbdddfb34efd25b49"
format_stamp: "Formatted at 2025-01-11 09:01:10 on dist-test-slave-nvbt"
I20250111 09:01:10.758706 14588 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.005s sys 0.000s
I20250111 09:01:10.760332 14602 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:10.761124 14588 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:01:10.761219 14588 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/master-0/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/master-0/wal
uuid: "28fc30b9e1394ddbbdddfb34efd25b49"
format_stamp: "Formatted at 2025-01-11 09:01:10 on dist-test-slave-nvbt"
I20250111 09:01:10.761281 14588 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:01:10.775300 14588 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:01:10.775728 14588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:01:10.775853 14588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:01:10.782370 14588 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:43521
I20250111 09:01:10.782378 14664 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:43521 every 8 connection(s)
I20250111 09:01:10.783164 14588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
I20250111 09:01:10.785295 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:10.788690 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 14588
I20250111 09:01:10.788808 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/master-0/wal/instance
I20250111 09:01:10.789076 14665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49: Bootstrap starting.
I20250111 09:01:10.790879 14665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:10.791517 14665 log.cc:826] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49: Log is configured to *not* fsync() on all Append() calls
I20250111 09:01:10.793315 14665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49: No bootstrap required, opened a new log
I20250111 09:01:10.796659 14665 raft_consensus.cc:357] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fc30b9e1394ddbbdddfb34efd25b49" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 43521 } }
I20250111 09:01:10.796820 14665 raft_consensus.cc:383] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:10.796864 14665 raft_consensus.cc:738] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28fc30b9e1394ddbbdddfb34efd25b49, State: Initialized, Role: FOLLOWER
I20250111 09:01:10.797235 14665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fc30b9e1394ddbbdddfb34efd25b49" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 43521 } }
I20250111 09:01:10.797328 14665 raft_consensus.cc:397] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:01:10.797384 14665 raft_consensus.cc:491] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:01:10.797475 14665 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:10.798045 14665 raft_consensus.cc:513] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fc30b9e1394ddbbdddfb34efd25b49" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 43521 } }
I20250111 09:01:10.798319 14665 leader_election.cc:304] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28fc30b9e1394ddbbdddfb34efd25b49; no voters:
I20250111 09:01:10.798534 14665 leader_election.cc:290] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:01:10.798643 14670 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:10.798832 14670 raft_consensus.cc:695] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49 [term 1 LEADER]: Becoming Leader. State: Replica: 28fc30b9e1394ddbbdddfb34efd25b49, State: Running, Role: LEADER
I20250111 09:01:10.799124 14670 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fc30b9e1394ddbbdddfb34efd25b49" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 43521 } }
I20250111 09:01:10.799528 14665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49 [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:01:10.800055 14670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28fc30b9e1394ddbbdddfb34efd25b49. Latest consensus state: current_term: 1 leader_uuid: "28fc30b9e1394ddbbdddfb34efd25b49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fc30b9e1394ddbbdddfb34efd25b49" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 43521 } } }
I20250111 09:01:10.800212 14670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49 [sys.catalog]: This master's current role is: LEADER
I20250111 09:01:10.800633 14671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28fc30b9e1394ddbbdddfb34efd25b49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fc30b9e1394ddbbdddfb34efd25b49" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 43521 } } }
I20250111 09:01:10.800829 14671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49 [sys.catalog]: This master's current role is: LEADER
I20250111 09:01:10.800637 14677 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:01:10.803594 14677 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:01:10.808059 14677 catalog_manager.cc:1348] Generated new cluster ID: 139c1b76ef184c6fa9c3613fab146d5b
I20250111 09:01:10.808106 14677 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:01:10.816704 14677 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:01:10.817624 14677 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:01:10.831869 14677 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 28fc30b9e1394ddbbdddfb34efd25b49: Generated new TSK 0
I20250111 09:01:10.832481 14677 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:01:10.839181 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.1:0
--local_ip_for_outbound_sockets=127.14.61.1
--webserver_interface=127.14.61.1
--webserver_port=0
--tserver_master_addrs=127.14.61.62:43521
--builtin_ntp_servers=127.14.61.20:36263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_txn_system_client_init=true with env {}
I20250111 09:01:10.841532 14618 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44288:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20250111 09:01:10.844583 14665 master.cc:440] Invalid argument: unable to initialize TxnManager: Error creating table kudu_system.kudu_transactions on the master: not enough live tablet servers to create a table with the requested replication factor 3; 0 tablet servers are alive: unable to init TxnManager, will retry in 1.000s
W20250111 09:01:10.944191 14694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:01:10.944358 14694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:01:10.944434 14694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:01:10.947609 14694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:01:10.947690 14694 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:01:10.947762 14694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.1
I20250111 09:01:10.950984 14694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:36263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.61.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:43521
--never_fsync=true
--heap_profile_path=/tmp/kudu.14694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.1
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:01:10.951812 14694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:01:10.952764 14694 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:01:10.960253 14699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:10.960312 14700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:10.961055 14702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:01:10.961694 14694 server_base.cc:1034] running on GCE node
I20250111 09:01:10.962038 14694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:01:10.962502 14694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:01:10.963644 14694 hybrid_clock.cc:648] HybridClock initialized: now 1736586070963632 us; error 32 us; skew 500 ppm
I20250111 09:01:10.965560 14694 webserver.cc:458] Webserver started at http://127.14.61.1:45443/ using document root <none> and password file <none>
I20250111 09:01:10.966086 14694 fs_manager.cc:362] Metadata directory not provided
I20250111 09:01:10.966135 14694 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:01:10.966334 14694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:01:10.967813 14694 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-0/data/instance:
uuid: "ad3147d5ba854715923b217158d9c4ba"
format_stamp: "Formatted at 2025-01-11 09:01:10 on dist-test-slave-nvbt"
I20250111 09:01:10.968230 14694 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-0/wal/instance:
uuid: "ad3147d5ba854715923b217158d9c4ba"
format_stamp: "Formatted at 2025-01-11 09:01:10 on dist-test-slave-nvbt"
I20250111 09:01:10.971185 14694 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20250111 09:01:10.973063 14708 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:10.973860 14694 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:01:10.973946 14694 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-0/wal
uuid: "ad3147d5ba854715923b217158d9c4ba"
format_stamp: "Formatted at 2025-01-11 09:01:10 on dist-test-slave-nvbt"
I20250111 09:01:10.974013 14694 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:01:10.995785 14694 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:01:10.996224 14694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:01:10.996378 14694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:01:10.997637 14694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:01:10.997691 14694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:10.997742 14694 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:01:10.997776 14694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:11.007627 14694 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:37461
I20250111 09:01:11.007669 14827 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:37461 every 8 connection(s)
I20250111 09:01:11.008412 14694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
I20250111 09:01:11.009425 14829 heartbeater.cc:346] Connected to a master server at 127.14.61.62:43521
I20250111 09:01:11.009534 14829 heartbeater.cc:463] Registering TS with master...
I20250111 09:01:11.009910 14829 heartbeater.cc:510] Master 127.14.61.62:43521 requested a full tablet report, sending...
I20250111 09:01:11.011176 14618 ts_manager.cc:194] Registered new tserver with Master: ad3147d5ba854715923b217158d9c4ba (127.14.61.1:37461)
I20250111 09:01:11.012385 14618 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.1:38233
I20250111 09:01:11.016752 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 14694
I20250111 09:01:11.016825 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-0/wal/instance
I20250111 09:01:11.018494 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.2:0
--local_ip_for_outbound_sockets=127.14.61.2
--webserver_interface=127.14.61.2
--webserver_port=0
--tserver_master_addrs=127.14.61.62:43521
--builtin_ntp_servers=127.14.61.20:36263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_txn_system_client_init=true with env {}
W20250111 09:01:11.106516 14831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:01:11.106730 14831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:01:11.106796 14831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:01:11.109339 14831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:01:11.109409 14831 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:01:11.109477 14831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.2
I20250111 09:01:11.112541 14831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:36263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.61.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:43521
--never_fsync=true
--heap_profile_path=/tmp/kudu.14831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.2
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:01:11.113339 14831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:01:11.114157 14831 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:01:11.120226 14836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:11.120368 14837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:11.120561 14839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:01:11.121263 14831 server_base.cc:1034] running on GCE node
I20250111 09:01:11.121500 14831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:01:11.121922 14831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:01:11.123116 14831 hybrid_clock.cc:648] HybridClock initialized: now 1736586071123095 us; error 21 us; skew 500 ppm
I20250111 09:01:11.124740 14831 webserver.cc:458] Webserver started at http://127.14.61.2:42267/ using document root <none> and password file <none>
I20250111 09:01:11.125241 14831 fs_manager.cc:362] Metadata directory not provided
I20250111 09:01:11.125294 14831 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:01:11.125468 14831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:01:11.126749 14831 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-1/data/instance:
uuid: "3d72c489d7ed49c683088673ea9465b6"
format_stamp: "Formatted at 2025-01-11 09:01:11 on dist-test-slave-nvbt"
I20250111 09:01:11.127110 14831 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-1/wal/instance:
uuid: "3d72c489d7ed49c683088673ea9465b6"
format_stamp: "Formatted at 2025-01-11 09:01:11 on dist-test-slave-nvbt"
I20250111 09:01:11.129626 14831 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20250111 09:01:11.131167 14845 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:11.131971 14831 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20250111 09:01:11.132066 14831 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-1/wal
uuid: "3d72c489d7ed49c683088673ea9465b6"
format_stamp: "Formatted at 2025-01-11 09:01:11 on dist-test-slave-nvbt"
I20250111 09:01:11.132135 14831 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:01:11.149668 14831 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:01:11.150036 14831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:01:11.150154 14831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:01:11.151207 14831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:01:11.151258 14831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:11.151298 14831 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:01:11.151327 14831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:11.160414 14831 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.2:43319
I20250111 09:01:11.160441 14965 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.2:43319 every 8 connection(s)
I20250111 09:01:11.161164 14831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
I20250111 09:01:11.162282 14966 heartbeater.cc:346] Connected to a master server at 127.14.61.62:43521
I20250111 09:01:11.162401 14966 heartbeater.cc:463] Registering TS with master...
I20250111 09:01:11.162855 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 14831
I20250111 09:01:11.162861 14966 heartbeater.cc:510] Master 127.14.61.62:43521 requested a full tablet report, sending...
I20250111 09:01:11.162940 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-1/wal/instance
I20250111 09:01:11.163724 14618 ts_manager.cc:194] Registered new tserver with Master: 3d72c489d7ed49c683088673ea9465b6 (127.14.61.2:43319)
I20250111 09:01:11.164274 14618 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.2:35825
I20250111 09:01:11.164939 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.3:0
--local_ip_for_outbound_sockets=127.14.61.3
--webserver_interface=127.14.61.3
--webserver_port=0
--tserver_master_addrs=127.14.61.62:43521
--builtin_ntp_servers=127.14.61.20:36263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_txn_system_client_init=true with env {}
W20250111 09:01:11.253918 14968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:01:11.254058 14968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:01:11.254094 14968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:01:11.256551 14968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:01:11.256608 14968 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:01:11.256644 14968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.3
I20250111 09:01:11.259632 14968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:36263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.61.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:43521
--never_fsync=true
--heap_profile_path=/tmp/kudu.14968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.3
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:01:11.260322 14968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:01:11.261129 14968 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:01:11.267590 14976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:11.267735 14973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:11.267942 14974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:01:11.268604 14968 server_base.cc:1034] running on GCE node
I20250111 09:01:11.268828 14968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:01:11.269241 14968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:01:11.270408 14968 hybrid_clock.cc:648] HybridClock initialized: now 1736586071270374 us; error 22 us; skew 500 ppm
I20250111 09:01:11.271759 14968 webserver.cc:458] Webserver started at http://127.14.61.3:34761/ using document root <none> and password file <none>
I20250111 09:01:11.272225 14968 fs_manager.cc:362] Metadata directory not provided
I20250111 09:01:11.272276 14968 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:01:11.272436 14968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:01:11.273767 14968 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-2/data/instance:
uuid: "d2dbead934e3475a8e83c26aca379d66"
format_stamp: "Formatted at 2025-01-11 09:01:11 on dist-test-slave-nvbt"
I20250111 09:01:11.274138 14968 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-2/wal/instance:
uuid: "d2dbead934e3475a8e83c26aca379d66"
format_stamp: "Formatted at 2025-01-11 09:01:11 on dist-test-slave-nvbt"
I20250111 09:01:11.276741 14968 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20250111 09:01:11.278515 14982 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:11.279366 14968 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20250111 09:01:11.279449 14968 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-2/wal
uuid: "d2dbead934e3475a8e83c26aca379d66"
format_stamp: "Formatted at 2025-01-11 09:01:11 on dist-test-slave-nvbt"
I20250111 09:01:11.279524 14968 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:01:11.293534 14968 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:01:11.293953 14968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:01:11.294091 14968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:01:11.295766 14968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:01:11.295872 14968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:11.295969 14968 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:01:11.296037 14968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:11.305884 14968 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.3:35339
I20250111 09:01:11.306864 14968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
I20250111 09:01:11.308351 15097 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.3:35339 every 8 connection(s)
I20250111 09:01:11.309386 15098 heartbeater.cc:346] Connected to a master server at 127.14.61.62:43521
I20250111 09:01:11.309530 15098 heartbeater.cc:463] Registering TS with master...
I20250111 09:01:11.309614 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 14968
I20250111 09:01:11.309716 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.CommitTimestampPropagation.1736586070622223-14580-0/minicluster-data/ts-2/wal/instance
I20250111 09:01:11.310143 15098 heartbeater.cc:510] Master 127.14.61.62:43521 requested a full tablet report, sending...
I20250111 09:01:11.310950 14618 ts_manager.cc:194] Registered new tserver with Master: d2dbead934e3475a8e83c26aca379d66 (127.14.61.3:35339)
I20250111 09:01:11.311482 14618 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.3:41699
I20250111 09:01:11.312323 14580 external_mini_cluster.cc:934] 3 TS(s) registered with all masters
I20250111 09:01:11.340452 14619 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44296:
name: "txn_write_ops_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
W20250111 09:01:11.341248 14619 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table txn_write_ops_test in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250111 09:01:11.357061 15021 tablet_service.cc:1467] Processing CreateTablet for tablet f68d37f649f14ccb869d265bed61efcd (DEFAULT_TABLE table=txn_write_ops_test [id=82b04ad4f03f4ef085c6732acc7dbed9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:11.357388 15022 tablet_service.cc:1467] Processing CreateTablet for tablet 4d6958ae00fb4e2eac6e0aef8b0f317a (DEFAULT_TABLE table=txn_write_ops_test [id=82b04ad4f03f4ef085c6732acc7dbed9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:11.357153 14894 tablet_service.cc:1467] Processing CreateTablet for tablet 4d6958ae00fb4e2eac6e0aef8b0f317a (DEFAULT_TABLE table=txn_write_ops_test [id=82b04ad4f03f4ef085c6732acc7dbed9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:11.357153 14895 tablet_service.cc:1467] Processing CreateTablet for tablet f68d37f649f14ccb869d265bed61efcd (DEFAULT_TABLE table=txn_write_ops_test [id=82b04ad4f03f4ef085c6732acc7dbed9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:11.357908 15022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d6958ae00fb4e2eac6e0aef8b0f317a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:11.357926 14894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d6958ae00fb4e2eac6e0aef8b0f317a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:11.358173 14895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f68d37f649f14ccb869d265bed61efcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:11.358222 15021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f68d37f649f14ccb869d265bed61efcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:11.362717 14758 tablet_service.cc:1467] Processing CreateTablet for tablet f68d37f649f14ccb869d265bed61efcd (DEFAULT_TABLE table=txn_write_ops_test [id=82b04ad4f03f4ef085c6732acc7dbed9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:11.362723 14757 tablet_service.cc:1467] Processing CreateTablet for tablet 4d6958ae00fb4e2eac6e0aef8b0f317a (DEFAULT_TABLE table=txn_write_ops_test [id=82b04ad4f03f4ef085c6732acc7dbed9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:11.363477 14758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f68d37f649f14ccb869d265bed61efcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:11.364610 14757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d6958ae00fb4e2eac6e0aef8b0f317a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:11.371050 15122 tablet_bootstrap.cc:492] T 4d6958ae00fb4e2eac6e0aef8b0f317a P d2dbead934e3475a8e83c26aca379d66: Bootstrap starting.
I20250111 09:01:11.373263 15121 tablet_bootstrap.cc:492] T f68d37f649f14ccb869d265bed61efcd P ad3147d5ba854715923b217158d9c4ba: Bootstrap starting.
I20250111 09:01:11.373880 15122 tablet_bootstrap.cc:654] T 4d6958ae00fb4e2eac6e0aef8b0f317a P d2dbead934e3475a8e83c26aca379d66: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:11.374692 15121 tablet_bootstrap.cc:654] T f68d37f649f14ccb869d265bed61efcd P ad3147d5ba854715923b217158d9c4ba: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:11.374675 15122 log.cc:826] T 4d6958ae00fb4e2eac6e0aef8b0f317a P d2dbead934e3475a8e83c26aca379d66: Log is configured to *not* fsync() on all Append() calls
I20250111 09:01:11.375340 15121 log.cc:826] T f68d37f649f14ccb869d265bed61efcd P ad3147d5ba854715923b217158d9c4ba: Log is configured to *not* fsync() on all Append() calls
I20250111 09:01:11.372983 15120 tablet_bootstrap.cc:492] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6: Bootstrap starting.
I20250111 09:01:11.376461 15122 tablet_bootstrap.cc:492] T 4d6958ae00fb4e2eac6e0aef8b0f317a P d2dbead934e3475a8e83c26aca379d66: No bootstrap required, opened a new log
I20250111 09:01:11.376627 15122 ts_tablet_manager.cc:1397] T 4d6958ae00fb4e2eac6e0aef8b0f317a P d2dbead934e3475a8e83c26aca379d66: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20250111 09:01:11.376973 15121 tablet_bootstrap.cc:492] T f68d37f649f14ccb869d265bed61efcd P ad3147d5ba854715923b217158d9c4ba: No bootstrap required, opened a new log
I20250111 09:01:11.377192 15121 ts_tablet_manager.cc:1397] T f68d37f649f14ccb869d265bed61efcd P ad3147d5ba854715923b217158d9c4ba: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20250111 09:01:11.377583 15120 tablet_bootstrap.cc:654] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:11.378450 15120 log.cc:826] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6: Log is configured to *not* fsync() on all Append() calls
I20250111 09:01:11.379907 15120 tablet_bootstrap.cc:492] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6: No bootstrap required, opened a new log
I20250111 09:01:11.380048 15120 ts_tablet_manager.cc:1397] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20250111 09:01:11.380373 15122 raft_consensus.cc:357] T 4d6958ae00fb4e2eac6e0aef8b0f317a P d2dbead934e3475a8e83c26aca379d66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.380573 15122 raft_consensus.cc:383] T 4d6958ae00fb4e2eac6e0aef8b0f317a P d2dbead934e3475a8e83c26aca379d66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:11.380622 15122 raft_consensus.cc:738] T 4d6958ae00fb4e2eac6e0aef8b0f317a P d2dbead934e3475a8e83c26aca379d66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2dbead934e3475a8e83c26aca379d66, State: Initialized, Role: FOLLOWER
I20250111 09:01:11.380920 15121 raft_consensus.cc:357] T f68d37f649f14ccb869d265bed61efcd P ad3147d5ba854715923b217158d9c4ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.381089 15121 raft_consensus.cc:383] T f68d37f649f14ccb869d265bed61efcd P ad3147d5ba854715923b217158d9c4ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:11.381129 15121 raft_consensus.cc:738] T f68d37f649f14ccb869d265bed61efcd P ad3147d5ba854715923b217158d9c4ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad3147d5ba854715923b217158d9c4ba, State: Initialized, Role: FOLLOWER
I20250111 09:01:11.381081 15122 consensus_queue.cc:260] T 4d6958ae00fb4e2eac6e0aef8b0f317a P d2dbead934e3475a8e83c26aca379d66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.381481 15121 consensus_queue.cc:260] T f68d37f649f14ccb869d265bed61efcd P ad3147d5ba854715923b217158d9c4ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.382016 15122 ts_tablet_manager.cc:1428] T 4d6958ae00fb4e2eac6e0aef8b0f317a P d2dbead934e3475a8e83c26aca379d66: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20250111 09:01:11.382246 15121 ts_tablet_manager.cc:1428] T f68d37f649f14ccb869d265bed61efcd P ad3147d5ba854715923b217158d9c4ba: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250111 09:01:11.382398 15098 heartbeater.cc:502] Master 127.14.61.62:43521 was elected leader, sending a full tablet report...
I20250111 09:01:11.382417 15122 tablet_bootstrap.cc:492] T f68d37f649f14ccb869d265bed61efcd P d2dbead934e3475a8e83c26aca379d66: Bootstrap starting.
I20250111 09:01:11.382536 14829 heartbeater.cc:502] Master 127.14.61.62:43521 was elected leader, sending a full tablet report...
I20250111 09:01:11.383635 15122 tablet_bootstrap.cc:654] T f68d37f649f14ccb869d265bed61efcd P d2dbead934e3475a8e83c26aca379d66: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:11.384110 15120 raft_consensus.cc:357] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.384296 15120 raft_consensus.cc:383] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:11.384346 15120 raft_consensus.cc:738] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d72c489d7ed49c683088673ea9465b6, State: Initialized, Role: FOLLOWER
I20250111 09:01:11.384761 15120 consensus_queue.cc:260] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.385424 14966 heartbeater.cc:502] Master 127.14.61.62:43521 was elected leader, sending a full tablet report...
I20250111 09:01:11.385543 15120 ts_tablet_manager.cc:1428] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20250111 09:01:11.385775 15120 tablet_bootstrap.cc:492] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6: Bootstrap starting.
I20250111 09:01:11.386726 15120 tablet_bootstrap.cc:654] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:11.387223 15121 tablet_bootstrap.cc:492] T 4d6958ae00fb4e2eac6e0aef8b0f317a P ad3147d5ba854715923b217158d9c4ba: Bootstrap starting.
I20250111 09:01:11.387708 15120 tablet_bootstrap.cc:492] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6: No bootstrap required, opened a new log
I20250111 09:01:11.387777 15120 ts_tablet_manager.cc:1397] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20250111 09:01:11.388387 15120 raft_consensus.cc:357] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.388482 15120 raft_consensus.cc:383] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:11.388512 15120 raft_consensus.cc:738] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d72c489d7ed49c683088673ea9465b6, State: Initialized, Role: FOLLOWER
I20250111 09:01:11.388624 15120 consensus_queue.cc:260] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.388810 15120 ts_tablet_manager.cc:1428] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:01:11.388945 15121 tablet_bootstrap.cc:654] T 4d6958ae00fb4e2eac6e0aef8b0f317a P ad3147d5ba854715923b217158d9c4ba: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:11.390159 15121 tablet_bootstrap.cc:492] T 4d6958ae00fb4e2eac6e0aef8b0f317a P ad3147d5ba854715923b217158d9c4ba: No bootstrap required, opened a new log
I20250111 09:01:11.390251 15121 ts_tablet_manager.cc:1397] T 4d6958ae00fb4e2eac6e0aef8b0f317a P ad3147d5ba854715923b217158d9c4ba: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250111 09:01:11.390623 15121 raft_consensus.cc:357] T 4d6958ae00fb4e2eac6e0aef8b0f317a P ad3147d5ba854715923b217158d9c4ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.390707 15121 raft_consensus.cc:383] T 4d6958ae00fb4e2eac6e0aef8b0f317a P ad3147d5ba854715923b217158d9c4ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:11.390738 15121 raft_consensus.cc:738] T 4d6958ae00fb4e2eac6e0aef8b0f317a P ad3147d5ba854715923b217158d9c4ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad3147d5ba854715923b217158d9c4ba, State: Initialized, Role: FOLLOWER
I20250111 09:01:11.390863 15121 consensus_queue.cc:260] T 4d6958ae00fb4e2eac6e0aef8b0f317a P ad3147d5ba854715923b217158d9c4ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.391094 15121 ts_tablet_manager.cc:1428] T 4d6958ae00fb4e2eac6e0aef8b0f317a P ad3147d5ba854715923b217158d9c4ba: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:01:11.392834 15122 tablet_bootstrap.cc:492] T f68d37f649f14ccb869d265bed61efcd P d2dbead934e3475a8e83c26aca379d66: No bootstrap required, opened a new log
I20250111 09:01:11.392930 15122 ts_tablet_manager.cc:1397] T f68d37f649f14ccb869d265bed61efcd P d2dbead934e3475a8e83c26aca379d66: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20250111 09:01:11.393433 15122 raft_consensus.cc:357] T f68d37f649f14ccb869d265bed61efcd P d2dbead934e3475a8e83c26aca379d66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.393543 15122 raft_consensus.cc:383] T f68d37f649f14ccb869d265bed61efcd P d2dbead934e3475a8e83c26aca379d66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:11.393582 15122 raft_consensus.cc:738] T f68d37f649f14ccb869d265bed61efcd P d2dbead934e3475a8e83c26aca379d66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2dbead934e3475a8e83c26aca379d66, State: Initialized, Role: FOLLOWER
I20250111 09:01:11.393710 15122 consensus_queue.cc:260] T f68d37f649f14ccb869d265bed61efcd P d2dbead934e3475a8e83c26aca379d66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.393990 15122 ts_tablet_manager.cc:1428] T f68d37f649f14ccb869d265bed61efcd P d2dbead934e3475a8e83c26aca379d66: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20250111 09:01:11.411893 14967 tablet.cc:2367] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250111 09:01:11.503142 15129 raft_consensus.cc:491] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:01:11.503284 15129 raft_consensus.cc:513] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.504165 15129 leader_election.cc:290] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad3147d5ba854715923b217158d9c4ba (127.14.61.1:37461), d2dbead934e3475a8e83c26aca379d66 (127.14.61.3:35339)
I20250111 09:01:11.507426 14777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d6958ae00fb4e2eac6e0aef8b0f317a" candidate_uuid: "3d72c489d7ed49c683088673ea9465b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad3147d5ba854715923b217158d9c4ba" is_pre_election: true
I20250111 09:01:11.507594 15035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d6958ae00fb4e2eac6e0aef8b0f317a" candidate_uuid: "3d72c489d7ed49c683088673ea9465b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2dbead934e3475a8e83c26aca379d66" is_pre_election: true
I20250111 09:01:11.507639 14777 raft_consensus.cc:2463] T 4d6958ae00fb4e2eac6e0aef8b0f317a P ad3147d5ba854715923b217158d9c4ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d72c489d7ed49c683088673ea9465b6 in term 0.
I20250111 09:01:11.507807 15035 raft_consensus.cc:2463] T 4d6958ae00fb4e2eac6e0aef8b0f317a P d2dbead934e3475a8e83c26aca379d66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d72c489d7ed49c683088673ea9465b6 in term 0.
I20250111 09:01:11.507987 14847 leader_election.cc:304] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d72c489d7ed49c683088673ea9465b6, ad3147d5ba854715923b217158d9c4ba; no voters:
I20250111 09:01:11.508172 15129 raft_consensus.cc:2798] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:01:11.508229 15129 raft_consensus.cc:491] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:01:11.508278 15129 raft_consensus.cc:3054] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:11.508994 15129 raft_consensus.cc:513] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.509253 15129 leader_election.cc:290] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6 [CANDIDATE]: Term 1 election: Requested vote from peers ad3147d5ba854715923b217158d9c4ba (127.14.61.1:37461), d2dbead934e3475a8e83c26aca379d66 (127.14.61.3:35339)
W20250111 09:01:11.509418 14830 tablet.cc:2367] T 4d6958ae00fb4e2eac6e0aef8b0f317a P ad3147d5ba854715923b217158d9c4ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250111 09:01:11.509559 15035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d6958ae00fb4e2eac6e0aef8b0f317a" candidate_uuid: "3d72c489d7ed49c683088673ea9465b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2dbead934e3475a8e83c26aca379d66"
I20250111 09:01:11.509649 15035 raft_consensus.cc:3054] T 4d6958ae00fb4e2eac6e0aef8b0f317a P d2dbead934e3475a8e83c26aca379d66 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:11.509970 14777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d6958ae00fb4e2eac6e0aef8b0f317a" candidate_uuid: "3d72c489d7ed49c683088673ea9465b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad3147d5ba854715923b217158d9c4ba"
I20250111 09:01:11.510046 14777 raft_consensus.cc:3054] T 4d6958ae00fb4e2eac6e0aef8b0f317a P ad3147d5ba854715923b217158d9c4ba [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:11.510461 15035 raft_consensus.cc:2463] T 4d6958ae00fb4e2eac6e0aef8b0f317a P d2dbead934e3475a8e83c26aca379d66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d72c489d7ed49c683088673ea9465b6 in term 1.
I20250111 09:01:11.510706 14847 leader_election.cc:304] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d72c489d7ed49c683088673ea9465b6, d2dbead934e3475a8e83c26aca379d66; no voters:
I20250111 09:01:11.510787 14777 raft_consensus.cc:2463] T 4d6958ae00fb4e2eac6e0aef8b0f317a P ad3147d5ba854715923b217158d9c4ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d72c489d7ed49c683088673ea9465b6 in term 1.
I20250111 09:01:11.510836 15129 raft_consensus.cc:2798] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:11.511008 15129 raft_consensus.cc:695] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6 [term 1 LEADER]: Becoming Leader. State: Replica: 3d72c489d7ed49c683088673ea9465b6, State: Running, Role: LEADER
I20250111 09:01:11.511294 15129 consensus_queue.cc:237] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.513877 14619 catalog_manager.cc:5526] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d72c489d7ed49c683088673ea9465b6 (127.14.61.2). New cstate: current_term: 1 leader_uuid: "3d72c489d7ed49c683088673ea9465b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:11.553398 15129 raft_consensus.cc:491] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:01:11.553534 15129 raft_consensus.cc:513] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.553761 15129 leader_election.cc:290] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad3147d5ba854715923b217158d9c4ba (127.14.61.1:37461), d2dbead934e3475a8e83c26aca379d66 (127.14.61.3:35339)
I20250111 09:01:11.554030 14777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f68d37f649f14ccb869d265bed61efcd" candidate_uuid: "3d72c489d7ed49c683088673ea9465b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad3147d5ba854715923b217158d9c4ba" is_pre_election: true
I20250111 09:01:11.554039 15035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f68d37f649f14ccb869d265bed61efcd" candidate_uuid: "3d72c489d7ed49c683088673ea9465b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2dbead934e3475a8e83c26aca379d66" is_pre_election: true
I20250111 09:01:11.554111 14777 raft_consensus.cc:2463] T f68d37f649f14ccb869d265bed61efcd P ad3147d5ba854715923b217158d9c4ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d72c489d7ed49c683088673ea9465b6 in term 0.
I20250111 09:01:11.554131 15035 raft_consensus.cc:2463] T f68d37f649f14ccb869d265bed61efcd P d2dbead934e3475a8e83c26aca379d66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d72c489d7ed49c683088673ea9465b6 in term 0.
I20250111 09:01:11.554349 14847 leader_election.cc:304] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d72c489d7ed49c683088673ea9465b6, ad3147d5ba854715923b217158d9c4ba; no voters:
I20250111 09:01:11.554499 15129 raft_consensus.cc:2798] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:01:11.554554 15129 raft_consensus.cc:491] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:01:11.554611 15129 raft_consensus.cc:3054] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:11.555263 15129 raft_consensus.cc:513] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.555503 15129 leader_election.cc:290] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6 [CANDIDATE]: Term 1 election: Requested vote from peers ad3147d5ba854715923b217158d9c4ba (127.14.61.1:37461), d2dbead934e3475a8e83c26aca379d66 (127.14.61.3:35339)
I20250111 09:01:11.555755 14777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f68d37f649f14ccb869d265bed61efcd" candidate_uuid: "3d72c489d7ed49c683088673ea9465b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad3147d5ba854715923b217158d9c4ba"
I20250111 09:01:11.555809 15035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f68d37f649f14ccb869d265bed61efcd" candidate_uuid: "3d72c489d7ed49c683088673ea9465b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2dbead934e3475a8e83c26aca379d66"
I20250111 09:01:11.555848 14777 raft_consensus.cc:3054] T f68d37f649f14ccb869d265bed61efcd P ad3147d5ba854715923b217158d9c4ba [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:11.555918 15035 raft_consensus.cc:3054] T f68d37f649f14ccb869d265bed61efcd P d2dbead934e3475a8e83c26aca379d66 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:11.556452 14777 raft_consensus.cc:2463] T f68d37f649f14ccb869d265bed61efcd P ad3147d5ba854715923b217158d9c4ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d72c489d7ed49c683088673ea9465b6 in term 1.
I20250111 09:01:11.556507 15035 raft_consensus.cc:2463] T f68d37f649f14ccb869d265bed61efcd P d2dbead934e3475a8e83c26aca379d66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d72c489d7ed49c683088673ea9465b6 in term 1.
I20250111 09:01:11.556694 14847 leader_election.cc:304] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d72c489d7ed49c683088673ea9465b6, ad3147d5ba854715923b217158d9c4ba; no voters:
I20250111 09:01:11.556818 15129 raft_consensus.cc:2798] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:11.556896 15129 raft_consensus.cc:695] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6 [term 1 LEADER]: Becoming Leader. State: Replica: 3d72c489d7ed49c683088673ea9465b6, State: Running, Role: LEADER
I20250111 09:01:11.556996 15129 consensus_queue.cc:237] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.558164 14619 catalog_manager.cc:5526] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d72c489d7ed49c683088673ea9465b6 (127.14.61.2). New cstate: current_term: 1 leader_uuid: "3d72c489d7ed49c683088673ea9465b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:11.559031 15099 tablet.cc:2367] T 4d6958ae00fb4e2eac6e0aef8b0f317a P d2dbead934e3475a8e83c26aca379d66: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250111 09:01:11.595150 15107 txn_manager_proxy_rpc.cc:150] re-attempting BeginTransaction request to one of TxnManagers
I20250111 09:01:11.850972 14619 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44300:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20250111 09:01:11.851472 14619 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250111 09:01:11.855479 14758 tablet_service.cc:1467] Processing CreateTablet for tablet 6c5ef593f60b40539c593691c8a5dbb6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ca21039ade2b4988ad1709f49bdc87ca]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:01:11.855530 14894 tablet_service.cc:1467] Processing CreateTablet for tablet 6c5ef593f60b40539c593691c8a5dbb6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ca21039ade2b4988ad1709f49bdc87ca]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:01:11.855775 15021 tablet_service.cc:1467] Processing CreateTablet for tablet 6c5ef593f60b40539c593691c8a5dbb6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ca21039ade2b4988ad1709f49bdc87ca]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:01:11.856067 14894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c5ef593f60b40539c593691c8a5dbb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:11.856072 14758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c5ef593f60b40539c593691c8a5dbb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:11.856174 15021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c5ef593f60b40539c593691c8a5dbb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:11.857955 15121 tablet_bootstrap.cc:492] T 6c5ef593f60b40539c593691c8a5dbb6 P ad3147d5ba854715923b217158d9c4ba: Bootstrap starting.
I20250111 09:01:11.858377 15120 tablet_bootstrap.cc:492] T 6c5ef593f60b40539c593691c8a5dbb6 P 3d72c489d7ed49c683088673ea9465b6: Bootstrap starting.
I20250111 09:01:11.858839 15121 tablet_bootstrap.cc:654] T 6c5ef593f60b40539c593691c8a5dbb6 P ad3147d5ba854715923b217158d9c4ba: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:11.859114 15120 tablet_bootstrap.cc:654] T 6c5ef593f60b40539c593691c8a5dbb6 P 3d72c489d7ed49c683088673ea9465b6: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:11.859649 15122 tablet_bootstrap.cc:492] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66: Bootstrap starting.
I20250111 09:01:11.859772 15121 tablet_bootstrap.cc:492] T 6c5ef593f60b40539c593691c8a5dbb6 P ad3147d5ba854715923b217158d9c4ba: No bootstrap required, opened a new log
I20250111 09:01:11.859845 15121 ts_tablet_manager.cc:1397] T 6c5ef593f60b40539c593691c8a5dbb6 P ad3147d5ba854715923b217158d9c4ba: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:01:11.860244 15121 raft_consensus.cc:357] T 6c5ef593f60b40539c593691c8a5dbb6 P ad3147d5ba854715923b217158d9c4ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.860327 15121 raft_consensus.cc:383] T 6c5ef593f60b40539c593691c8a5dbb6 P ad3147d5ba854715923b217158d9c4ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:11.860363 15121 raft_consensus.cc:738] T 6c5ef593f60b40539c593691c8a5dbb6 P ad3147d5ba854715923b217158d9c4ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad3147d5ba854715923b217158d9c4ba, State: Initialized, Role: FOLLOWER
I20250111 09:01:11.860435 15122 tablet_bootstrap.cc:654] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:11.860467 15121 consensus_queue.cc:260] T 6c5ef593f60b40539c593691c8a5dbb6 P ad3147d5ba854715923b217158d9c4ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.860735 15121 ts_tablet_manager.cc:1428] T 6c5ef593f60b40539c593691c8a5dbb6 P ad3147d5ba854715923b217158d9c4ba: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:01:11.860700 15127 tablet_replica.cc:440] T 6c5ef593f60b40539c593691c8a5dbb6 P ad3147d5ba854715923b217158d9c4ba: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } } }
I20250111 09:01:11.860803 15127 tablet_replica.cc:443] T 6c5ef593f60b40539c593691c8a5dbb6 P ad3147d5ba854715923b217158d9c4ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:11.861467 15120 tablet_bootstrap.cc:492] T 6c5ef593f60b40539c593691c8a5dbb6 P 3d72c489d7ed49c683088673ea9465b6: No bootstrap required, opened a new log
I20250111 09:01:11.861544 15120 ts_tablet_manager.cc:1397] T 6c5ef593f60b40539c593691c8a5dbb6 P 3d72c489d7ed49c683088673ea9465b6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250111 09:01:11.861595 15122 tablet_bootstrap.cc:492] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66: No bootstrap required, opened a new log
I20250111 09:01:11.861665 15122 ts_tablet_manager.cc:1397] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:01:11.862000 15120 raft_consensus.cc:357] T 6c5ef593f60b40539c593691c8a5dbb6 P 3d72c489d7ed49c683088673ea9465b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.862109 15120 raft_consensus.cc:383] T 6c5ef593f60b40539c593691c8a5dbb6 P 3d72c489d7ed49c683088673ea9465b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:11.862149 15120 raft_consensus.cc:738] T 6c5ef593f60b40539c593691c8a5dbb6 P 3d72c489d7ed49c683088673ea9465b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d72c489d7ed49c683088673ea9465b6, State: Initialized, Role: FOLLOWER
I20250111 09:01:11.862097 15122 raft_consensus.cc:357] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.862191 15122 raft_consensus.cc:383] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:11.862223 15122 raft_consensus.cc:738] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2dbead934e3475a8e83c26aca379d66, State: Initialized, Role: FOLLOWER
I20250111 09:01:11.862272 15120 consensus_queue.cc:260] T 6c5ef593f60b40539c593691c8a5dbb6 P 3d72c489d7ed49c683088673ea9465b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.862324 15122 consensus_queue.cc:260] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.862524 15120 ts_tablet_manager.cc:1428] T 6c5ef593f60b40539c593691c8a5dbb6 P 3d72c489d7ed49c683088673ea9465b6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:01:11.862569 15122 ts_tablet_manager.cc:1428] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250111 09:01:11.862749 15129 tablet_replica.cc:440] T 6c5ef593f60b40539c593691c8a5dbb6 P 3d72c489d7ed49c683088673ea9465b6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } } }
I20250111 09:01:11.862872 15129 tablet_replica.cc:443] T 6c5ef593f60b40539c593691c8a5dbb6 P 3d72c489d7ed49c683088673ea9465b6: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:11.863052 15126 tablet_replica.cc:440] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } } }
I20250111 09:01:11.863145 15126 tablet_replica.cc:443] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:11.939059 15126 raft_consensus.cc:491] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:01:11.939148 15126 raft_consensus.cc:513] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.940032 15126 leader_election.cc:290] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad3147d5ba854715923b217158d9c4ba (127.14.61.1:37461), 3d72c489d7ed49c683088673ea9465b6 (127.14.61.2:43319)
I20250111 09:01:11.943228 14915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c5ef593f60b40539c593691c8a5dbb6" candidate_uuid: "d2dbead934e3475a8e83c26aca379d66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d72c489d7ed49c683088673ea9465b6" is_pre_election: true
I20250111 09:01:11.943246 14777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c5ef593f60b40539c593691c8a5dbb6" candidate_uuid: "d2dbead934e3475a8e83c26aca379d66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad3147d5ba854715923b217158d9c4ba" is_pre_election: true
I20250111 09:01:11.943353 14777 raft_consensus.cc:2463] T 6c5ef593f60b40539c593691c8a5dbb6 P ad3147d5ba854715923b217158d9c4ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2dbead934e3475a8e83c26aca379d66 in term 0.
I20250111 09:01:11.943396 14915 raft_consensus.cc:2463] T 6c5ef593f60b40539c593691c8a5dbb6 P 3d72c489d7ed49c683088673ea9465b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2dbead934e3475a8e83c26aca379d66 in term 0.
I20250111 09:01:11.943691 14984 leader_election.cc:304] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad3147d5ba854715923b217158d9c4ba, d2dbead934e3475a8e83c26aca379d66; no voters:
I20250111 09:01:11.943921 15126 raft_consensus.cc:2798] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:01:11.943974 15126 raft_consensus.cc:491] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:01:11.944015 15126 raft_consensus.cc:3054] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:11.944689 15126 raft_consensus.cc:513] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.944981 15126 leader_election.cc:290] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66 [CANDIDATE]: Term 1 election: Requested vote from peers ad3147d5ba854715923b217158d9c4ba (127.14.61.1:37461), 3d72c489d7ed49c683088673ea9465b6 (127.14.61.2:43319)
I20250111 09:01:11.945326 14915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c5ef593f60b40539c593691c8a5dbb6" candidate_uuid: "d2dbead934e3475a8e83c26aca379d66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d72c489d7ed49c683088673ea9465b6"
I20250111 09:01:11.945326 14777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c5ef593f60b40539c593691c8a5dbb6" candidate_uuid: "d2dbead934e3475a8e83c26aca379d66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad3147d5ba854715923b217158d9c4ba"
I20250111 09:01:11.945430 14915 raft_consensus.cc:3054] T 6c5ef593f60b40539c593691c8a5dbb6 P 3d72c489d7ed49c683088673ea9465b6 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:11.945456 14777 raft_consensus.cc:3054] T 6c5ef593f60b40539c593691c8a5dbb6 P ad3147d5ba854715923b217158d9c4ba [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:11.946333 14915 raft_consensus.cc:2463] T 6c5ef593f60b40539c593691c8a5dbb6 P 3d72c489d7ed49c683088673ea9465b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2dbead934e3475a8e83c26aca379d66 in term 1.
I20250111 09:01:11.946333 14777 raft_consensus.cc:2463] T 6c5ef593f60b40539c593691c8a5dbb6 P ad3147d5ba854715923b217158d9c4ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2dbead934e3475a8e83c26aca379d66 in term 1.
I20250111 09:01:11.946590 14984 leader_election.cc:304] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad3147d5ba854715923b217158d9c4ba, d2dbead934e3475a8e83c26aca379d66; no voters:
I20250111 09:01:11.946736 15126 raft_consensus.cc:2798] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:11.946892 15126 raft_consensus.cc:695] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66 [term 1 LEADER]: Becoming Leader. State: Replica: d2dbead934e3475a8e83c26aca379d66, State: Running, Role: LEADER
I20250111 09:01:11.947147 15126 consensus_queue.cc:237] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } }
I20250111 09:01:11.948632 15148 tablet_replica.cc:440] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66: TxnStatusTablet state changed. Reason: New leader d2dbead934e3475a8e83c26aca379d66. Latest consensus state: current_term: 1 leader_uuid: "d2dbead934e3475a8e83c26aca379d66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } } }
I20250111 09:01:11.948721 15148 tablet_replica.cc:443] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:11.948956 15150 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:01:11.949148 15150 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:01:11.949721 14617 catalog_manager.cc:5526] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2dbead934e3475a8e83c26aca379d66 (127.14.61.3). New cstate: current_term: 1 leader_uuid: "d2dbead934e3475a8e83c26aca379d66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:11.960414 15129 consensus_queue.cc:1035] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:11.965029 15154 consensus_queue.cc:1035] T f68d37f649f14ccb869d265bed61efcd P 3d72c489d7ed49c683088673ea9465b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:12.004906 15154 consensus_queue.cc:1035] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:12.005337 14915 raft_consensus.cc:1270] T 6c5ef593f60b40539c593691c8a5dbb6 P 3d72c489d7ed49c683088673ea9465b6 [term 1 FOLLOWER]: Refusing update from remote peer d2dbead934e3475a8e83c26aca379d66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:12.005693 14778 raft_consensus.cc:1270] T 6c5ef593f60b40539c593691c8a5dbb6 P ad3147d5ba854715923b217158d9c4ba [term 1 FOLLOWER]: Refusing update from remote peer d2dbead934e3475a8e83c26aca379d66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:12.006110 15126 consensus_queue.cc:1035] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:12.006408 15148 consensus_queue.cc:1035] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:12.008792 15127 tablet_replica.cc:440] T 6c5ef593f60b40539c593691c8a5dbb6 P ad3147d5ba854715923b217158d9c4ba: TxnStatusTablet state changed. Reason: New leader d2dbead934e3475a8e83c26aca379d66. Latest consensus state: current_term: 1 leader_uuid: "d2dbead934e3475a8e83c26aca379d66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } } }
I20250111 09:01:12.008957 15127 tablet_replica.cc:443] T 6c5ef593f60b40539c593691c8a5dbb6 P ad3147d5ba854715923b217158d9c4ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:12.009964 15133 tablet_replica.cc:440] T 6c5ef593f60b40539c593691c8a5dbb6 P 3d72c489d7ed49c683088673ea9465b6: TxnStatusTablet state changed. Reason: New leader d2dbead934e3475a8e83c26aca379d66. Latest consensus state: current_term: 1 leader_uuid: "d2dbead934e3475a8e83c26aca379d66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } } }
I20250111 09:01:12.010046 15133 tablet_replica.cc:443] T 6c5ef593f60b40539c593691c8a5dbb6 P 3d72c489d7ed49c683088673ea9465b6: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:12.011583 15154 consensus_queue.cc:1035] T 4d6958ae00fb4e2eac6e0aef8b0f317a P 3d72c489d7ed49c683088673ea9465b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:12.012890 15126 tablet_replica.cc:440] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d2dbead934e3475a8e83c26aca379d66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } } }
I20250111 09:01:12.012981 15126 tablet_replica.cc:443] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:12.013083 15126 tablet_replica.cc:440] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d2dbead934e3475a8e83c26aca379d66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } } }
I20250111 09:01:12.013132 15126 tablet_replica.cc:443] T 6c5ef593f60b40539c593691c8a5dbb6 P d2dbead934e3475a8e83c26aca379d66: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:12.013375 15155 mvcc.cc:204] Tried to move back new op lower bound from 7113056550928752640 to 7113056550697705472. Current Snapshot: MvccSnapshot[applied={T|T < 7113056550928752640}]
I20250111 09:01:12.013481 15127 tablet_replica.cc:440] T 6c5ef593f60b40539c593691c8a5dbb6 P ad3147d5ba854715923b217158d9c4ba: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d2dbead934e3475a8e83c26aca379d66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } } }
I20250111 09:01:12.013545 15127 tablet_replica.cc:443] T 6c5ef593f60b40539c593691c8a5dbb6 P ad3147d5ba854715923b217158d9c4ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:12.014600 15154 tablet_replica.cc:440] T 6c5ef593f60b40539c593691c8a5dbb6 P 3d72c489d7ed49c683088673ea9465b6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d2dbead934e3475a8e83c26aca379d66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3147d5ba854715923b217158d9c4ba" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37461 } } peers { permanent_uuid: "3d72c489d7ed49c683088673ea9465b6" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 43319 } } peers { permanent_uuid: "d2dbead934e3475a8e83c26aca379d66" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35339 } } }
I20250111 09:01:12.014674 15154 tablet_replica.cc:443] T 6c5ef593f60b40539c593691c8a5dbb6 P 3d72c489d7ed49c683088673ea9465b6: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:12.015249 15157 mvcc.cc:204] Tried to move back new op lower bound from 7113056550928752640 to 7113056550697705472. Current Snapshot: MvccSnapshot[applied={T|T < 7113056550928752640}]
I20250111 09:01:12.015393 15161 mvcc.cc:204] Tried to move back new op lower bound from 7113056550928752640 to 7113056550697705472. Current Snapshot: MvccSnapshot[applied={T|T < 7113056550928752640}]
I20250111 09:01:12.399345 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 14694
I20250111 09:01:12.407732 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 14831
I20250111 09:01:12.415839 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 14968
I20250111 09:01:12.422168 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 14588
2025-01-11T09:01:12Z chronyd exiting
[ OK ] TxnWriteOpsITest.CommitTimestampPropagation (1814 ms)
[ RUN ] TxnWriteOpsITest.DeadlockPrevention
2025-01-11T09:01:12Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-01-11T09:01:12Z Disabled control of system clock
I20250111 09:01:12.451951 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.61.62:42847
--webserver_interface=127.14.61.62
--webserver_port=0
--builtin_ntp_servers=127.14.61.20:35689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.61.62:42847
--txn_manager_enabled=true
--txn_manager_lazily_initialized=false with env {}
W20250111 09:01:12.541800 15189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:01:12.541981 15189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:01:12.542023 15189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:01:12.544767 15189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250111 09:01:12.544819 15189 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20250111 09:01:12.544839 15189 flags.cc:425] Enabled experimental flag: --txn_manager_lazily_initialized=false
W20250111 09:01:12.544858 15189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:01:12.544895 15189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250111 09:01:12.544924 15189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250111 09:01:12.548069 15189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:35689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.61.62:42847
--txn_manager_enabled=true
--txn_manager_lazily_initialized=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.61.62:42847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.61.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.15189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:01:12.548789 15189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:01:12.549631 15189 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:01:12.555474 15197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:12.555631 15194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:12.555738 15195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:01:12.556511 15189 server_base.cc:1034] running on GCE node
I20250111 09:01:12.556875 15189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:01:12.557569 15189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:01:12.558708 15189 hybrid_clock.cc:648] HybridClock initialized: now 1736586072558689 us; error 32 us; skew 500 ppm
I20250111 09:01:12.560148 15189 webserver.cc:458] Webserver started at http://127.14.61.62:36789/ using document root <none> and password file <none>
I20250111 09:01:12.560563 15189 fs_manager.cc:362] Metadata directory not provided
I20250111 09:01:12.560621 15189 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:01:12.560777 15189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:01:12.562136 15189 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/master-0/data/instance:
uuid: "0c8a81aae45048fc94b23117a63c2e3f"
format_stamp: "Formatted at 2025-01-11 09:01:12 on dist-test-slave-nvbt"
I20250111 09:01:12.562556 15189 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/master-0/wal/instance:
uuid: "0c8a81aae45048fc94b23117a63c2e3f"
format_stamp: "Formatted at 2025-01-11 09:01:12 on dist-test-slave-nvbt"
I20250111 09:01:12.565287 15189 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20250111 09:01:12.566972 15203 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:12.567745 15189 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20250111 09:01:12.567834 15189 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/master-0/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/master-0/wal
uuid: "0c8a81aae45048fc94b23117a63c2e3f"
format_stamp: "Formatted at 2025-01-11 09:01:12 on dist-test-slave-nvbt"
I20250111 09:01:12.567899 15189 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:01:12.579704 15189 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:01:12.580114 15189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:01:12.580237 15189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:01:12.586324 15189 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:42847
I20250111 09:01:12.586328 15265 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:42847 every 8 connection(s)
I20250111 09:01:12.587047 15189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
I20250111 09:01:12.588992 15266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:12.592890 15266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f: Bootstrap starting.
I20250111 09:01:12.594623 15266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:12.595160 15266 log.cc:826] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f: Log is configured to *not* fsync() on all Append() calls
I20250111 09:01:12.595676 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 15189
I20250111 09:01:12.595779 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/master-0/wal/instance
I20250111 09:01:12.596614 15266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f: No bootstrap required, opened a new log
I20250111 09:01:12.599337 15266 raft_consensus.cc:357] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a81aae45048fc94b23117a63c2e3f" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 42847 } }
I20250111 09:01:12.599468 15266 raft_consensus.cc:383] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:12.599498 15266 raft_consensus.cc:738] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c8a81aae45048fc94b23117a63c2e3f, State: Initialized, Role: FOLLOWER
I20250111 09:01:12.599830 15266 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a81aae45048fc94b23117a63c2e3f" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 42847 } }
I20250111 09:01:12.599936 15266 raft_consensus.cc:397] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:01:12.599989 15266 raft_consensus.cc:491] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:01:12.600078 15266 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:12.600672 15266 raft_consensus.cc:513] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a81aae45048fc94b23117a63c2e3f" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 42847 } }
I20250111 09:01:12.600943 15266 leader_election.cc:304] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c8a81aae45048fc94b23117a63c2e3f; no voters:
I20250111 09:01:12.601178 15266 leader_election.cc:290] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:01:12.601267 15271 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:12.601440 15271 raft_consensus.cc:695] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f [term 1 LEADER]: Becoming Leader. State: Replica: 0c8a81aae45048fc94b23117a63c2e3f, State: Running, Role: LEADER
I20250111 09:01:12.601691 15271 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a81aae45048fc94b23117a63c2e3f" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 42847 } }
I20250111 09:01:12.602109 15266 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:01:12.602749 15271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c8a81aae45048fc94b23117a63c2e3f. Latest consensus state: current_term: 1 leader_uuid: "0c8a81aae45048fc94b23117a63c2e3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a81aae45048fc94b23117a63c2e3f" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 42847 } } }
I20250111 09:01:12.602870 15271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f [sys.catalog]: This master's current role is: LEADER
I20250111 09:01:12.603144 15273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c8a81aae45048fc94b23117a63c2e3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a81aae45048fc94b23117a63c2e3f" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 42847 } } }
I20250111 09:01:12.603224 15273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f [sys.catalog]: This master's current role is: LEADER
W20250111 09:01:12.606055 15286 catalog_manager.cc:1559] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250111 09:01:12.606122 15286 catalog_manager.cc:874] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250111 09:01:12.606190 15279 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:01:12.606983 15279 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:01:12.611310 15279 catalog_manager.cc:1348] Generated new cluster ID: 1bd44c94456848c495a1b043eb766df3
I20250111 09:01:12.611359 15279 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:01:12.619256 15279 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:01:12.619915 15279 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:01:12.626609 15279 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 0c8a81aae45048fc94b23117a63c2e3f: Generated new TSK 0
I20250111 09:01:12.627072 15279 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:01:12.631114 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.1:0
--local_ip_for_outbound_sockets=127.14.61.1
--webserver_interface=127.14.61.1
--webserver_port=0
--tserver_master_addrs=127.14.61.62:42847
--builtin_ntp_servers=127.14.61.20:35689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_txn_system_client_init=true with env {}
I20250111 09:01:12.633688 15219 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53642:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20250111 09:01:12.638540 15266 master.cc:440] Invalid argument: unable to initialize TxnManager: Error creating table kudu_system.kudu_transactions on the master: not enough live tablet servers to create a table with the requested replication factor 3; 0 tablet servers are alive: unable to init TxnManager, will retry in 1.000s
W20250111 09:01:12.722573 15295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:01:12.722767 15295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:01:12.722841 15295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:01:12.725617 15295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:01:12.725705 15295 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:01:12.725778 15295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.1
I20250111 09:01:12.728782 15295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:35689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.61.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:42847
--never_fsync=true
--heap_profile_path=/tmp/kudu.15295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.1
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:01:12.729574 15295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:01:12.730460 15295 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:01:12.737596 15303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:12.738106 15300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:12.738273 15301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:01:12.738925 15295 server_base.cc:1034] running on GCE node
I20250111 09:01:12.739194 15295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:01:12.739648 15295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:01:12.740788 15295 hybrid_clock.cc:648] HybridClock initialized: now 1736586072740773 us; error 31 us; skew 500 ppm
I20250111 09:01:12.742429 15295 webserver.cc:458] Webserver started at http://127.14.61.1:33007/ using document root <none> and password file <none>
I20250111 09:01:12.742938 15295 fs_manager.cc:362] Metadata directory not provided
I20250111 09:01:12.743014 15295 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:01:12.743153 15295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:01:12.744513 15295 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-0/data/instance:
uuid: "c7db184c57854a4fadfb4502edf6f999"
format_stamp: "Formatted at 2025-01-11 09:01:12 on dist-test-slave-nvbt"
I20250111 09:01:12.744961 15295 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-0/wal/instance:
uuid: "c7db184c57854a4fadfb4502edf6f999"
format_stamp: "Formatted at 2025-01-11 09:01:12 on dist-test-slave-nvbt"
I20250111 09:01:12.747838 15295 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20250111 09:01:12.749549 15309 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:12.750324 15295 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20250111 09:01:12.750412 15295 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-0/wal
uuid: "c7db184c57854a4fadfb4502edf6f999"
format_stamp: "Formatted at 2025-01-11 09:01:12 on dist-test-slave-nvbt"
I20250111 09:01:12.750479 15295 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:01:12.767748 15295 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:01:12.768184 15295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:01:12.768306 15295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:01:12.769556 15295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:01:12.769623 15295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:12.769670 15295 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:01:12.769699 15295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:12.779101 15295 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:43947
I20250111 09:01:12.779127 15429 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:43947 every 8 connection(s)
I20250111 09:01:12.779937 15295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
I20250111 09:01:12.780985 15430 heartbeater.cc:346] Connected to a master server at 127.14.61.62:42847
I20250111 09:01:12.781347 15430 heartbeater.cc:463] Registering TS with master...
I20250111 09:01:12.781800 15430 heartbeater.cc:510] Master 127.14.61.62:42847 requested a full tablet report, sending...
I20250111 09:01:12.782992 15219 ts_manager.cc:194] Registered new tserver with Master: c7db184c57854a4fadfb4502edf6f999 (127.14.61.1:43947)
I20250111 09:01:12.784193 15219 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.1:49673
I20250111 09:01:12.786325 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 15295
I20250111 09:01:12.786409 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-0/wal/instance
I20250111 09:01:12.787950 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.2:0
--local_ip_for_outbound_sockets=127.14.61.2
--webserver_interface=127.14.61.2
--webserver_port=0
--tserver_master_addrs=127.14.61.62:42847
--builtin_ntp_servers=127.14.61.20:35689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_txn_system_client_init=true with env {}
W20250111 09:01:12.878136 15432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:01:12.878334 15432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:01:12.878405 15432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:01:12.881026 15432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:01:12.881085 15432 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:01:12.881121 15432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.2
I20250111 09:01:12.884248 15432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:35689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.61.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:42847
--never_fsync=true
--heap_profile_path=/tmp/kudu.15432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.2
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:01:12.885023 15432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:01:12.885874 15432 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:01:12.891953 15440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:12.892609 15438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:12.892829 15437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:01:12.893474 15432 server_base.cc:1034] running on GCE node
I20250111 09:01:12.893740 15432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:01:12.894163 15432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:01:12.895303 15432 hybrid_clock.cc:648] HybridClock initialized: now 1736586072895285 us; error 30 us; skew 500 ppm
I20250111 09:01:12.896684 15432 webserver.cc:458] Webserver started at http://127.14.61.2:39403/ using document root <none> and password file <none>
I20250111 09:01:12.897138 15432 fs_manager.cc:362] Metadata directory not provided
I20250111 09:01:12.897185 15432 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:01:12.897329 15432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:01:12.898538 15432 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-1/data/instance:
uuid: "ee65a36c423b48658c01089b07a24c3d"
format_stamp: "Formatted at 2025-01-11 09:01:12 on dist-test-slave-nvbt"
I20250111 09:01:12.898918 15432 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-1/wal/instance:
uuid: "ee65a36c423b48658c01089b07a24c3d"
format_stamp: "Formatted at 2025-01-11 09:01:12 on dist-test-slave-nvbt"
I20250111 09:01:12.901567 15432 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20250111 09:01:12.903384 15446 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:12.904170 15432 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250111 09:01:12.904251 15432 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-1/wal
uuid: "ee65a36c423b48658c01089b07a24c3d"
format_stamp: "Formatted at 2025-01-11 09:01:12 on dist-test-slave-nvbt"
I20250111 09:01:12.904318 15432 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:01:12.917274 15432 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:01:12.917701 15432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:01:12.917820 15432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:01:12.918946 15432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:01:12.919008 15432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:12.919056 15432 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:01:12.919093 15432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:12.928016 15432 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.2:45229
I20250111 09:01:12.928043 15565 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.2:45229 every 8 connection(s)
I20250111 09:01:12.928684 15432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
I20250111 09:01:12.929720 15566 heartbeater.cc:346] Connected to a master server at 127.14.61.62:42847
I20250111 09:01:12.929828 15566 heartbeater.cc:463] Registering TS with master...
I20250111 09:01:12.930222 15566 heartbeater.cc:510] Master 127.14.61.62:42847 requested a full tablet report, sending...
I20250111 09:01:12.930984 15219 ts_manager.cc:194] Registered new tserver with Master: ee65a36c423b48658c01089b07a24c3d (127.14.61.2:45229)
I20250111 09:01:12.931491 15219 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.2:35865
I20250111 09:01:12.933396 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 15432
I20250111 09:01:12.933495 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-1/wal/instance
I20250111 09:01:12.935185 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.3:0
--local_ip_for_outbound_sockets=127.14.61.3
--webserver_interface=127.14.61.3
--webserver_port=0
--tserver_master_addrs=127.14.61.62:42847
--builtin_ntp_servers=127.14.61.20:35689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_txn_system_client_init=true with env {}
W20250111 09:01:13.025306 15569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:01:13.025457 15569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:01:13.025498 15569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:01:13.028249 15569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:01:13.028306 15569 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:01:13.028339 15569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.3
I20250111 09:01:13.031414 15569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:35689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.61.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:42847
--never_fsync=true
--heap_profile_path=/tmp/kudu.15569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.3
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:01:13.032200 15569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:01:13.033110 15569 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:01:13.039290 15574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:13.039984 15575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:13.040165 15577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:01:13.040807 15569 server_base.cc:1034] running on GCE node
I20250111 09:01:13.041103 15569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:01:13.041508 15569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:01:13.042652 15569 hybrid_clock.cc:648] HybridClock initialized: now 1736586073042634 us; error 33 us; skew 500 ppm
I20250111 09:01:13.044204 15569 webserver.cc:458] Webserver started at http://127.14.61.3:43265/ using document root <none> and password file <none>
I20250111 09:01:13.044705 15569 fs_manager.cc:362] Metadata directory not provided
I20250111 09:01:13.044763 15569 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:01:13.044919 15569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:01:13.046378 15569 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-2/data/instance:
uuid: "0ad9dba2f090456ea13fa9cbea077682"
format_stamp: "Formatted at 2025-01-11 09:01:13 on dist-test-slave-nvbt"
I20250111 09:01:13.046770 15569 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-2/wal/instance:
uuid: "0ad9dba2f090456ea13fa9cbea077682"
format_stamp: "Formatted at 2025-01-11 09:01:13 on dist-test-slave-nvbt"
I20250111 09:01:13.049504 15569 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20250111 09:01:13.051184 15583 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:13.052027 15569 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:01:13.052120 15569 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-2/wal
uuid: "0ad9dba2f090456ea13fa9cbea077682"
format_stamp: "Formatted at 2025-01-11 09:01:13 on dist-test-slave-nvbt"
I20250111 09:01:13.052194 15569 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:01:13.072180 15569 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:01:13.072651 15569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:01:13.072782 15569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:01:13.073940 15569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:01:13.073992 15569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:13.074031 15569 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:01:13.074056 15569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:13.084116 15569 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.3:46065
I20250111 09:01:13.084362 15702 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.3:46065 every 8 connection(s)
I20250111 09:01:13.085067 15569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
I20250111 09:01:13.086079 15704 heartbeater.cc:346] Connected to a master server at 127.14.61.62:42847
I20250111 09:01:13.086184 15704 heartbeater.cc:463] Registering TS with master...
I20250111 09:01:13.086585 15704 heartbeater.cc:510] Master 127.14.61.62:42847 requested a full tablet report, sending...
I20250111 09:01:13.087309 15219 ts_manager.cc:194] Registered new tserver with Master: 0ad9dba2f090456ea13fa9cbea077682 (127.14.61.3:46065)
I20250111 09:01:13.087749 15219 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.3:45301
I20250111 09:01:13.089566 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 15569
I20250111 09:01:13.089665 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.DeadlockPrevention.1736586070622223-14580-0/minicluster-data/ts-2/wal/instance
I20250111 09:01:13.091297 14580 external_mini_cluster.cc:934] 3 TS(s) registered with all masters
I20250111 09:01:13.097898 15219 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53646:
name: "txn_write_ops_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
W20250111 09:01:13.098456 15219 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table txn_write_ops_test in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250111 09:01:13.110606 15359 tablet_service.cc:1467] Processing CreateTablet for tablet 41b32a69149146839fe96ce9b5e0f246 (DEFAULT_TABLE table=txn_write_ops_test [id=ff144c5c4d4e4ca0b1a5fe97a7d0016f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:13.110595 15358 tablet_service.cc:1467] Processing CreateTablet for tablet f85169e5db994f3690e7cdd2ff16c75a (DEFAULT_TABLE table=txn_write_ops_test [id=ff144c5c4d4e4ca0b1a5fe97a7d0016f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:13.111507 15358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f85169e5db994f3690e7cdd2ff16c75a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:13.112874 15495 tablet_service.cc:1467] Processing CreateTablet for tablet f85169e5db994f3690e7cdd2ff16c75a (DEFAULT_TABLE table=txn_write_ops_test [id=ff144c5c4d4e4ca0b1a5fe97a7d0016f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:13.112890 15496 tablet_service.cc:1467] Processing CreateTablet for tablet 41b32a69149146839fe96ce9b5e0f246 (DEFAULT_TABLE table=txn_write_ops_test [id=ff144c5c4d4e4ca0b1a5fe97a7d0016f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:13.113651 15359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41b32a69149146839fe96ce9b5e0f246. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:13.113770 15496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41b32a69149146839fe96ce9b5e0f246. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:13.115280 15628 tablet_service.cc:1467] Processing CreateTablet for tablet 41b32a69149146839fe96ce9b5e0f246 (DEFAULT_TABLE table=txn_write_ops_test [id=ff144c5c4d4e4ca0b1a5fe97a7d0016f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:13.116019 15628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41b32a69149146839fe96ce9b5e0f246. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:13.116380 15495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f85169e5db994f3690e7cdd2ff16c75a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:13.117790 15717 tablet_bootstrap.cc:492] T f85169e5db994f3690e7cdd2ff16c75a P c7db184c57854a4fadfb4502edf6f999: Bootstrap starting.
I20250111 09:01:13.119638 15718 tablet_bootstrap.cc:492] T 41b32a69149146839fe96ce9b5e0f246 P ee65a36c423b48658c01089b07a24c3d: Bootstrap starting.
I20250111 09:01:13.119733 15717 tablet_bootstrap.cc:654] T f85169e5db994f3690e7cdd2ff16c75a P c7db184c57854a4fadfb4502edf6f999: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:13.120410 15717 log.cc:826] T f85169e5db994f3690e7cdd2ff16c75a P c7db184c57854a4fadfb4502edf6f999: Log is configured to *not* fsync() on all Append() calls
I20250111 09:01:13.120750 15719 tablet_bootstrap.cc:492] T 41b32a69149146839fe96ce9b5e0f246 P 0ad9dba2f090456ea13fa9cbea077682: Bootstrap starting.
I20250111 09:01:13.121244 15626 tablet_service.cc:1467] Processing CreateTablet for tablet f85169e5db994f3690e7cdd2ff16c75a (DEFAULT_TABLE table=txn_write_ops_test [id=ff144c5c4d4e4ca0b1a5fe97a7d0016f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:13.121446 15626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f85169e5db994f3690e7cdd2ff16c75a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:13.121919 15717 tablet_bootstrap.cc:492] T f85169e5db994f3690e7cdd2ff16c75a P c7db184c57854a4fadfb4502edf6f999: No bootstrap required, opened a new log
I20250111 09:01:13.122071 15717 ts_tablet_manager.cc:1397] T f85169e5db994f3690e7cdd2ff16c75a P c7db184c57854a4fadfb4502edf6f999: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20250111 09:01:13.122323 15718 tablet_bootstrap.cc:654] T 41b32a69149146839fe96ce9b5e0f246 P ee65a36c423b48658c01089b07a24c3d: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:13.122992 15719 tablet_bootstrap.cc:654] T 41b32a69149146839fe96ce9b5e0f246 P 0ad9dba2f090456ea13fa9cbea077682: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:13.123042 15718 log.cc:826] T 41b32a69149146839fe96ce9b5e0f246 P ee65a36c423b48658c01089b07a24c3d: Log is configured to *not* fsync() on all Append() calls
I20250111 09:01:13.123643 15719 log.cc:826] T 41b32a69149146839fe96ce9b5e0f246 P 0ad9dba2f090456ea13fa9cbea077682: Log is configured to *not* fsync() on all Append() calls
I20250111 09:01:13.124437 15718 tablet_bootstrap.cc:492] T 41b32a69149146839fe96ce9b5e0f246 P ee65a36c423b48658c01089b07a24c3d: No bootstrap required, opened a new log
I20250111 09:01:13.124634 15718 ts_tablet_manager.cc:1397] T 41b32a69149146839fe96ce9b5e0f246 P ee65a36c423b48658c01089b07a24c3d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20250111 09:01:13.124732 15719 tablet_bootstrap.cc:492] T 41b32a69149146839fe96ce9b5e0f246 P 0ad9dba2f090456ea13fa9cbea077682: No bootstrap required, opened a new log
I20250111 09:01:13.124837 15719 ts_tablet_manager.cc:1397] T 41b32a69149146839fe96ce9b5e0f246 P 0ad9dba2f090456ea13fa9cbea077682: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20250111 09:01:13.125088 15717 raft_consensus.cc:357] T f85169e5db994f3690e7cdd2ff16c75a P c7db184c57854a4fadfb4502edf6f999 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } }
I20250111 09:01:13.125280 15717 raft_consensus.cc:383] T f85169e5db994f3690e7cdd2ff16c75a P c7db184c57854a4fadfb4502edf6f999 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:13.125345 15717 raft_consensus.cc:738] T f85169e5db994f3690e7cdd2ff16c75a P c7db184c57854a4fadfb4502edf6f999 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7db184c57854a4fadfb4502edf6f999, State: Initialized, Role: FOLLOWER
I20250111 09:01:13.125984 15717 consensus_queue.cc:260] T f85169e5db994f3690e7cdd2ff16c75a P c7db184c57854a4fadfb4502edf6f999 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } }
I20250111 09:01:13.126610 15430 heartbeater.cc:502] Master 127.14.61.62:42847 was elected leader, sending a full tablet report...
I20250111 09:01:13.126762 15717 ts_tablet_manager.cc:1428] T f85169e5db994f3690e7cdd2ff16c75a P c7db184c57854a4fadfb4502edf6f999: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20250111 09:01:13.126968 15717 tablet_bootstrap.cc:492] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999: Bootstrap starting.
I20250111 09:01:13.127617 15719 raft_consensus.cc:357] T 41b32a69149146839fe96ce9b5e0f246 P 0ad9dba2f090456ea13fa9cbea077682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } }
I20250111 09:01:13.127784 15719 raft_consensus.cc:383] T 41b32a69149146839fe96ce9b5e0f246 P 0ad9dba2f090456ea13fa9cbea077682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:13.127804 15717 tablet_bootstrap.cc:654] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:13.127835 15719 raft_consensus.cc:738] T 41b32a69149146839fe96ce9b5e0f246 P 0ad9dba2f090456ea13fa9cbea077682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad9dba2f090456ea13fa9cbea077682, State: Initialized, Role: FOLLOWER
I20250111 09:01:13.127753 15718 raft_consensus.cc:357] T 41b32a69149146839fe96ce9b5e0f246 P ee65a36c423b48658c01089b07a24c3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } }
I20250111 09:01:13.127943 15718 raft_consensus.cc:383] T 41b32a69149146839fe96ce9b5e0f246 P ee65a36c423b48658c01089b07a24c3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:13.127992 15718 raft_consensus.cc:738] T 41b32a69149146839fe96ce9b5e0f246 P ee65a36c423b48658c01089b07a24c3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee65a36c423b48658c01089b07a24c3d, State: Initialized, Role: FOLLOWER
I20250111 09:01:13.128196 15719 consensus_queue.cc:260] T 41b32a69149146839fe96ce9b5e0f246 P 0ad9dba2f090456ea13fa9cbea077682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } }
I20250111 09:01:13.128389 15718 consensus_queue.cc:260] T 41b32a69149146839fe96ce9b5e0f246 P ee65a36c423b48658c01089b07a24c3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } }
I20250111 09:01:13.128758 15717 tablet_bootstrap.cc:492] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999: No bootstrap required, opened a new log
I20250111 09:01:13.128818 15717 ts_tablet_manager.cc:1397] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20250111 09:01:13.128930 15719 ts_tablet_manager.cc:1428] T 41b32a69149146839fe96ce9b5e0f246 P 0ad9dba2f090456ea13fa9cbea077682: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20250111 09:01:13.129096 15718 ts_tablet_manager.cc:1428] T 41b32a69149146839fe96ce9b5e0f246 P ee65a36c423b48658c01089b07a24c3d: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20250111 09:01:13.129251 15704 heartbeater.cc:502] Master 127.14.61.62:42847 was elected leader, sending a full tablet report...
I20250111 09:01:13.129227 15717 raft_consensus.cc:357] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } }
I20250111 09:01:13.129310 15717 raft_consensus.cc:383] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:13.129336 15717 raft_consensus.cc:738] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7db184c57854a4fadfb4502edf6f999, State: Initialized, Role: FOLLOWER
I20250111 09:01:13.129402 15566 heartbeater.cc:502] Master 127.14.61.62:42847 was elected leader, sending a full tablet report...
I20250111 09:01:13.129442 15717 consensus_queue.cc:260] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } }
I20250111 09:01:13.129747 15717 ts_tablet_manager.cc:1428] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:01:13.130044 15719 tablet_bootstrap.cc:492] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682: Bootstrap starting.
I20250111 09:01:13.130364 15718 tablet_bootstrap.cc:492] T f85169e5db994f3690e7cdd2ff16c75a P ee65a36c423b48658c01089b07a24c3d: Bootstrap starting.
I20250111 09:01:13.130939 15719 tablet_bootstrap.cc:654] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:13.131098 15718 tablet_bootstrap.cc:654] T f85169e5db994f3690e7cdd2ff16c75a P ee65a36c423b48658c01089b07a24c3d: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:13.132715 15718 tablet_bootstrap.cc:492] T f85169e5db994f3690e7cdd2ff16c75a P ee65a36c423b48658c01089b07a24c3d: No bootstrap required, opened a new log
I20250111 09:01:13.132786 15718 ts_tablet_manager.cc:1397] T f85169e5db994f3690e7cdd2ff16c75a P ee65a36c423b48658c01089b07a24c3d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:01:13.133364 15718 raft_consensus.cc:357] T f85169e5db994f3690e7cdd2ff16c75a P ee65a36c423b48658c01089b07a24c3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } }
I20250111 09:01:13.133450 15718 raft_consensus.cc:383] T f85169e5db994f3690e7cdd2ff16c75a P ee65a36c423b48658c01089b07a24c3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:13.133481 15718 raft_consensus.cc:738] T f85169e5db994f3690e7cdd2ff16c75a P ee65a36c423b48658c01089b07a24c3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee65a36c423b48658c01089b07a24c3d, State: Initialized, Role: FOLLOWER
I20250111 09:01:13.133596 15718 consensus_queue.cc:260] T f85169e5db994f3690e7cdd2ff16c75a P ee65a36c423b48658c01089b07a24c3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } }
I20250111 09:01:13.133831 15718 ts_tablet_manager.cc:1428] T f85169e5db994f3690e7cdd2ff16c75a P ee65a36c423b48658c01089b07a24c3d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250111 09:01:13.133934 15719 tablet_bootstrap.cc:492] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682: No bootstrap required, opened a new log
I20250111 09:01:13.134009 15719 ts_tablet_manager.cc:1397] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250111 09:01:13.134341 15719 raft_consensus.cc:357] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } }
I20250111 09:01:13.134433 15719 raft_consensus.cc:383] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:13.134452 15719 raft_consensus.cc:738] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad9dba2f090456ea13fa9cbea077682, State: Initialized, Role: FOLLOWER
I20250111 09:01:13.134565 15719 consensus_queue.cc:260] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } }
I20250111 09:01:13.134815 15719 ts_tablet_manager.cc:1428] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20250111 09:01:13.179386 15567 tablet.cc:2367] T 41b32a69149146839fe96ce9b5e0f246 P ee65a36c423b48658c01089b07a24c3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250111 09:01:13.194161 15725 raft_consensus.cc:491] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:01:13.194326 15725 raft_consensus.cc:513] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } }
I20250111 09:01:13.195632 15725 leader_election.cc:290] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7db184c57854a4fadfb4502edf6f999 (127.14.61.1:43947), ee65a36c423b48658c01089b07a24c3d (127.14.61.2:45229)
I20250111 09:01:13.198933 15516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f85169e5db994f3690e7cdd2ff16c75a" candidate_uuid: "0ad9dba2f090456ea13fa9cbea077682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee65a36c423b48658c01089b07a24c3d" is_pre_election: true
I20250111 09:01:13.199191 15516 raft_consensus.cc:2463] T f85169e5db994f3690e7cdd2ff16c75a P ee65a36c423b48658c01089b07a24c3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ad9dba2f090456ea13fa9cbea077682 in term 0.
I20250111 09:01:13.199591 15584 leader_election.cc:304] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad9dba2f090456ea13fa9cbea077682, ee65a36c423b48658c01089b07a24c3d; no voters:
I20250111 09:01:13.199671 15379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f85169e5db994f3690e7cdd2ff16c75a" candidate_uuid: "0ad9dba2f090456ea13fa9cbea077682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7db184c57854a4fadfb4502edf6f999" is_pre_election: true
I20250111 09:01:13.199816 15725 raft_consensus.cc:2798] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:01:13.199908 15725 raft_consensus.cc:491] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:01:13.199929 15379 raft_consensus.cc:2463] T f85169e5db994f3690e7cdd2ff16c75a P c7db184c57854a4fadfb4502edf6f999 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ad9dba2f090456ea13fa9cbea077682 in term 0.
I20250111 09:01:13.199959 15725 raft_consensus.cc:3054] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:13.200767 15725 raft_consensus.cc:513] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } }
I20250111 09:01:13.201074 15725 leader_election.cc:290] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682 [CANDIDATE]: Term 1 election: Requested vote from peers c7db184c57854a4fadfb4502edf6f999 (127.14.61.1:43947), ee65a36c423b48658c01089b07a24c3d (127.14.61.2:45229)
I20250111 09:01:13.201357 15379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f85169e5db994f3690e7cdd2ff16c75a" candidate_uuid: "0ad9dba2f090456ea13fa9cbea077682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7db184c57854a4fadfb4502edf6f999"
I20250111 09:01:13.201431 15516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f85169e5db994f3690e7cdd2ff16c75a" candidate_uuid: "0ad9dba2f090456ea13fa9cbea077682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee65a36c423b48658c01089b07a24c3d"
I20250111 09:01:13.201462 15379 raft_consensus.cc:3054] T f85169e5db994f3690e7cdd2ff16c75a P c7db184c57854a4fadfb4502edf6f999 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:13.201539 15516 raft_consensus.cc:3054] T f85169e5db994f3690e7cdd2ff16c75a P ee65a36c423b48658c01089b07a24c3d [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:13.202296 15379 raft_consensus.cc:2463] T f85169e5db994f3690e7cdd2ff16c75a P c7db184c57854a4fadfb4502edf6f999 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad9dba2f090456ea13fa9cbea077682 in term 1.
I20250111 09:01:13.202375 15516 raft_consensus.cc:2463] T f85169e5db994f3690e7cdd2ff16c75a P ee65a36c423b48658c01089b07a24c3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad9dba2f090456ea13fa9cbea077682 in term 1.
I20250111 09:01:13.202570 15586 leader_election.cc:304] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad9dba2f090456ea13fa9cbea077682, c7db184c57854a4fadfb4502edf6f999; no voters:
I20250111 09:01:13.202713 15725 raft_consensus.cc:2798] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:13.202919 15725 raft_consensus.cc:695] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682 [term 1 LEADER]: Becoming Leader. State: Replica: 0ad9dba2f090456ea13fa9cbea077682, State: Running, Role: LEADER
I20250111 09:01:13.203176 15725 consensus_queue.cc:237] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } }
I20250111 09:01:13.206051 15220 catalog_manager.cc:5526] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ad9dba2f090456ea13fa9cbea077682 (127.14.61.3). New cstate: current_term: 1 leader_uuid: "0ad9dba2f090456ea13fa9cbea077682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:13.240078 15727 raft_consensus.cc:491] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:01:13.240190 15727 raft_consensus.cc:513] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } }
I20250111 09:01:13.241366 15727 leader_election.cc:290] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ad9dba2f090456ea13fa9cbea077682 (127.14.61.3:46065), ee65a36c423b48658c01089b07a24c3d (127.14.61.2:45229)
I20250111 09:01:13.246129 15516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b32a69149146839fe96ce9b5e0f246" candidate_uuid: "c7db184c57854a4fadfb4502edf6f999" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee65a36c423b48658c01089b07a24c3d" is_pre_election: true
I20250111 09:01:13.246268 15516 raft_consensus.cc:2463] T 41b32a69149146839fe96ce9b5e0f246 P ee65a36c423b48658c01089b07a24c3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7db184c57854a4fadfb4502edf6f999 in term 0.
I20250111 09:01:13.246274 15653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b32a69149146839fe96ce9b5e0f246" candidate_uuid: "c7db184c57854a4fadfb4502edf6f999" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad9dba2f090456ea13fa9cbea077682" is_pre_election: true
I20250111 09:01:13.246456 15653 raft_consensus.cc:2463] T 41b32a69149146839fe96ce9b5e0f246 P 0ad9dba2f090456ea13fa9cbea077682 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7db184c57854a4fadfb4502edf6f999 in term 0.
I20250111 09:01:13.246851 15310 leader_election.cc:304] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad9dba2f090456ea13fa9cbea077682, c7db184c57854a4fadfb4502edf6f999; no voters:
I20250111 09:01:13.247072 15727 raft_consensus.cc:2798] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:01:13.247129 15727 raft_consensus.cc:491] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:01:13.247165 15727 raft_consensus.cc:3054] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:13.247927 15727 raft_consensus.cc:513] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } }
I20250111 09:01:13.248206 15727 leader_election.cc:290] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999 [CANDIDATE]: Term 1 election: Requested vote from peers 0ad9dba2f090456ea13fa9cbea077682 (127.14.61.3:46065), ee65a36c423b48658c01089b07a24c3d (127.14.61.2:45229)
I20250111 09:01:13.248581 15653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b32a69149146839fe96ce9b5e0f246" candidate_uuid: "c7db184c57854a4fadfb4502edf6f999" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad9dba2f090456ea13fa9cbea077682"
I20250111 09:01:13.248667 15653 raft_consensus.cc:3054] T 41b32a69149146839fe96ce9b5e0f246 P 0ad9dba2f090456ea13fa9cbea077682 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:13.248633 15516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b32a69149146839fe96ce9b5e0f246" candidate_uuid: "c7db184c57854a4fadfb4502edf6f999" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee65a36c423b48658c01089b07a24c3d"
I20250111 09:01:13.248736 15516 raft_consensus.cc:3054] T 41b32a69149146839fe96ce9b5e0f246 P ee65a36c423b48658c01089b07a24c3d [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:13.249428 15516 raft_consensus.cc:2463] T 41b32a69149146839fe96ce9b5e0f246 P ee65a36c423b48658c01089b07a24c3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7db184c57854a4fadfb4502edf6f999 in term 1.
I20250111 09:01:13.249521 15653 raft_consensus.cc:2463] T 41b32a69149146839fe96ce9b5e0f246 P 0ad9dba2f090456ea13fa9cbea077682 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7db184c57854a4fadfb4502edf6f999 in term 1.
I20250111 09:01:13.249668 15310 leader_election.cc:304] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7db184c57854a4fadfb4502edf6f999, ee65a36c423b48658c01089b07a24c3d; no voters:
I20250111 09:01:13.249804 15727 raft_consensus.cc:2798] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:13.249905 15727 raft_consensus.cc:695] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999 [term 1 LEADER]: Becoming Leader. State: Replica: c7db184c57854a4fadfb4502edf6f999, State: Running, Role: LEADER
I20250111 09:01:13.250152 15727 consensus_queue.cc:237] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } }
I20250111 09:01:13.252699 15220 catalog_manager.cc:5526] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7db184c57854a4fadfb4502edf6f999 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "c7db184c57854a4fadfb4502edf6f999" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:13.270085 14580 test_util.cc:274] Using random seed: 1946478400
W20250111 09:01:13.271217 15708 txn_manager_proxy_rpc.cc:150] re-attempting BeginTransaction request to one of TxnManagers
W20250111 09:01:13.280784 15431 tablet.cc:2367] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250111 09:01:13.335845 15705 tablet.cc:2367] T 41b32a69149146839fe96ce9b5e0f246 P 0ad9dba2f090456ea13fa9cbea077682: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250111 09:01:13.645159 15220 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53656:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20250111 09:01:13.645603 15220 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250111 09:01:13.649658 15358 tablet_service.cc:1467] Processing CreateTablet for tablet 7044bb69c6e342ceb77cbcfe8b92c4b4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9f3af126a6434a01a9bfd38b664c2030]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:01:13.649716 15496 tablet_service.cc:1467] Processing CreateTablet for tablet 7044bb69c6e342ceb77cbcfe8b92c4b4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9f3af126a6434a01a9bfd38b664c2030]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:01:13.649896 15628 tablet_service.cc:1467] Processing CreateTablet for tablet 7044bb69c6e342ceb77cbcfe8b92c4b4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9f3af126a6434a01a9bfd38b664c2030]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:01:13.650243 15358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7044bb69c6e342ceb77cbcfe8b92c4b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:13.650242 15496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7044bb69c6e342ceb77cbcfe8b92c4b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:13.650368 15628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7044bb69c6e342ceb77cbcfe8b92c4b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:13.652813 15751 tablet_bootstrap.cc:492] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d: Bootstrap starting.
I20250111 09:01:13.653061 15750 tablet_bootstrap.cc:492] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P 0ad9dba2f090456ea13fa9cbea077682: Bootstrap starting.
I20250111 09:01:13.653766 15751 tablet_bootstrap.cc:654] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:13.653890 15750 tablet_bootstrap.cc:654] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P 0ad9dba2f090456ea13fa9cbea077682: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:13.653903 15727 consensus_queue.cc:1035] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:13.654837 15750 tablet_bootstrap.cc:492] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P 0ad9dba2f090456ea13fa9cbea077682: No bootstrap required, opened a new log
I20250111 09:01:13.654927 15750 ts_tablet_manager.cc:1397] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P 0ad9dba2f090456ea13fa9cbea077682: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:01:13.655336 15750 raft_consensus.cc:357] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P 0ad9dba2f090456ea13fa9cbea077682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } }
I20250111 09:01:13.655453 15750 raft_consensus.cc:383] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P 0ad9dba2f090456ea13fa9cbea077682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:13.655494 15750 raft_consensus.cc:738] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P 0ad9dba2f090456ea13fa9cbea077682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad9dba2f090456ea13fa9cbea077682, State: Initialized, Role: FOLLOWER
I20250111 09:01:13.655546 15751 tablet_bootstrap.cc:492] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d: No bootstrap required, opened a new log
I20250111 09:01:13.655634 15751 ts_tablet_manager.cc:1397] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250111 09:01:13.655601 15750 consensus_queue.cc:260] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P 0ad9dba2f090456ea13fa9cbea077682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } }
I20250111 09:01:13.655813 15725 tablet_replica.cc:440] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P 0ad9dba2f090456ea13fa9cbea077682: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } } }
I20250111 09:01:13.655932 15725 tablet_replica.cc:443] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P 0ad9dba2f090456ea13fa9cbea077682: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:13.656004 15752 tablet_bootstrap.cc:492] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P c7db184c57854a4fadfb4502edf6f999: Bootstrap starting.
I20250111 09:01:13.655870 15750 ts_tablet_manager.cc:1428] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P 0ad9dba2f090456ea13fa9cbea077682: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:01:13.658216 15752 tablet_bootstrap.cc:654] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P c7db184c57854a4fadfb4502edf6f999: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:13.659154 15727 consensus_queue.cc:1035] T 41b32a69149146839fe96ce9b5e0f246 P c7db184c57854a4fadfb4502edf6f999 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:13.659647 15751 raft_consensus.cc:357] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } }
I20250111 09:01:13.659749 15751 raft_consensus.cc:383] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:13.659790 15751 raft_consensus.cc:738] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee65a36c423b48658c01089b07a24c3d, State: Initialized, Role: FOLLOWER
I20250111 09:01:13.659914 15751 consensus_queue.cc:260] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } }
I20250111 09:01:13.659957 15752 tablet_bootstrap.cc:492] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P c7db184c57854a4fadfb4502edf6f999: No bootstrap required, opened a new log
I20250111 09:01:13.660027 15752 ts_tablet_manager.cc:1397] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P c7db184c57854a4fadfb4502edf6f999: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20250111 09:01:13.660116 15755 tablet_replica.cc:440] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } } }
I20250111 09:01:13.660223 15755 tablet_replica.cc:443] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:13.660399 15752 raft_consensus.cc:357] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P c7db184c57854a4fadfb4502edf6f999 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } }
I20250111 09:01:13.660483 15752 raft_consensus.cc:383] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P c7db184c57854a4fadfb4502edf6f999 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:13.660513 15752 raft_consensus.cc:738] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P c7db184c57854a4fadfb4502edf6f999 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7db184c57854a4fadfb4502edf6f999, State: Initialized, Role: FOLLOWER
I20250111 09:01:13.660619 15751 ts_tablet_manager.cc:1428] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20250111 09:01:13.660629 15752 consensus_queue.cc:260] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P c7db184c57854a4fadfb4502edf6f999 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } }
I20250111 09:01:13.660820 15727 tablet_replica.cc:440] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P c7db184c57854a4fadfb4502edf6f999: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } } }
I20250111 09:01:13.660920 15727 tablet_replica.cc:443] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P c7db184c57854a4fadfb4502edf6f999: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:13.661365 15752 ts_tablet_manager.cc:1428] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P c7db184c57854a4fadfb4502edf6f999: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:01:13.695196 15755 raft_consensus.cc:491] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:01:13.695277 15755 raft_consensus.cc:513] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } }
I20250111 09:01:13.696020 15755 leader_election.cc:290] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7db184c57854a4fadfb4502edf6f999 (127.14.61.1:43947), 0ad9dba2f090456ea13fa9cbea077682 (127.14.61.3:46065)
I20250111 09:01:13.699256 15379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7044bb69c6e342ceb77cbcfe8b92c4b4" candidate_uuid: "ee65a36c423b48658c01089b07a24c3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7db184c57854a4fadfb4502edf6f999" is_pre_election: true
I20250111 09:01:13.699363 15379 raft_consensus.cc:2463] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P c7db184c57854a4fadfb4502edf6f999 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee65a36c423b48658c01089b07a24c3d in term 0.
I20250111 09:01:13.699381 15653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7044bb69c6e342ceb77cbcfe8b92c4b4" candidate_uuid: "ee65a36c423b48658c01089b07a24c3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad9dba2f090456ea13fa9cbea077682" is_pre_election: true
I20250111 09:01:13.699508 15653 raft_consensus.cc:2463] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P 0ad9dba2f090456ea13fa9cbea077682 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee65a36c423b48658c01089b07a24c3d in term 0.
I20250111 09:01:13.699632 15449 leader_election.cc:304] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7db184c57854a4fadfb4502edf6f999, ee65a36c423b48658c01089b07a24c3d; no voters:
I20250111 09:01:13.699815 15755 raft_consensus.cc:2798] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:01:13.699865 15755 raft_consensus.cc:491] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:01:13.699908 15755 raft_consensus.cc:3054] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:13.700501 15755 raft_consensus.cc:513] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } }
I20250111 09:01:13.700747 15755 leader_election.cc:290] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d [CANDIDATE]: Term 1 election: Requested vote from peers c7db184c57854a4fadfb4502edf6f999 (127.14.61.1:43947), 0ad9dba2f090456ea13fa9cbea077682 (127.14.61.3:46065)
I20250111 09:01:13.701032 15653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7044bb69c6e342ceb77cbcfe8b92c4b4" candidate_uuid: "ee65a36c423b48658c01089b07a24c3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad9dba2f090456ea13fa9cbea077682"
I20250111 09:01:13.701031 15379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7044bb69c6e342ceb77cbcfe8b92c4b4" candidate_uuid: "ee65a36c423b48658c01089b07a24c3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7db184c57854a4fadfb4502edf6f999"
I20250111 09:01:13.701119 15379 raft_consensus.cc:3054] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P c7db184c57854a4fadfb4502edf6f999 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:13.701119 15653 raft_consensus.cc:3054] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P 0ad9dba2f090456ea13fa9cbea077682 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:13.701834 15379 raft_consensus.cc:2463] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P c7db184c57854a4fadfb4502edf6f999 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee65a36c423b48658c01089b07a24c3d in term 1.
I20250111 09:01:13.701834 15653 raft_consensus.cc:2463] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P 0ad9dba2f090456ea13fa9cbea077682 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee65a36c423b48658c01089b07a24c3d in term 1.
I20250111 09:01:13.702126 15447 leader_election.cc:304] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad9dba2f090456ea13fa9cbea077682, ee65a36c423b48658c01089b07a24c3d; no voters:
I20250111 09:01:13.702277 15755 raft_consensus.cc:2798] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:13.702451 15755 raft_consensus.cc:695] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d [term 1 LEADER]: Becoming Leader. State: Replica: ee65a36c423b48658c01089b07a24c3d, State: Running, Role: LEADER
I20250111 09:01:13.702730 15755 consensus_queue.cc:237] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } }
I20250111 09:01:13.703805 15764 tablet_replica.cc:440] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d: TxnStatusTablet state changed. Reason: New leader ee65a36c423b48658c01089b07a24c3d. Latest consensus state: current_term: 1 leader_uuid: "ee65a36c423b48658c01089b07a24c3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } } }
I20250111 09:01:13.703881 15764 tablet_replica.cc:443] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:13.704107 15766 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:01:13.704214 15766 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:01:13.704838 15218 catalog_manager.cc:5526] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d reported cstate change: term changed from 0 to 1, leader changed from <none> to ee65a36c423b48658c01089b07a24c3d (127.14.61.2). New cstate: current_term: 1 leader_uuid: "ee65a36c423b48658c01089b07a24c3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:13.708631 15725 consensus_queue.cc:1035] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:13.711758 15769 consensus_queue.cc:1035] T f85169e5db994f3690e7cdd2ff16c75a P 0ad9dba2f090456ea13fa9cbea077682 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:13.758677 15379 raft_consensus.cc:1270] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P c7db184c57854a4fadfb4502edf6f999 [term 1 FOLLOWER]: Refusing update from remote peer ee65a36c423b48658c01089b07a24c3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:13.758718 15653 raft_consensus.cc:1270] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P 0ad9dba2f090456ea13fa9cbea077682 [term 1 FOLLOWER]: Refusing update from remote peer ee65a36c423b48658c01089b07a24c3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:13.759126 15764 consensus_queue.cc:1035] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:13.759296 15755 consensus_queue.cc:1035] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:13.761478 15723 tablet_replica.cc:440] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P c7db184c57854a4fadfb4502edf6f999: TxnStatusTablet state changed. Reason: New leader ee65a36c423b48658c01089b07a24c3d. Latest consensus state: current_term: 1 leader_uuid: "ee65a36c423b48658c01089b07a24c3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } } }
I20250111 09:01:13.761606 15723 tablet_replica.cc:443] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P c7db184c57854a4fadfb4502edf6f999: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:13.761639 15769 tablet_replica.cc:440] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P 0ad9dba2f090456ea13fa9cbea077682: TxnStatusTablet state changed. Reason: New leader ee65a36c423b48658c01089b07a24c3d. Latest consensus state: current_term: 1 leader_uuid: "ee65a36c423b48658c01089b07a24c3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } } }
I20250111 09:01:13.761711 15769 tablet_replica.cc:443] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P 0ad9dba2f090456ea13fa9cbea077682: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:13.763942 15764 tablet_replica.cc:440] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ee65a36c423b48658c01089b07a24c3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } } }
I20250111 09:01:13.764031 15764 tablet_replica.cc:443] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:13.764199 15758 mvcc.cc:204] Tried to move back new op lower bound from 7113056558110900224 to 7113056557888303104. Current Snapshot: MvccSnapshot[applied={T|T < 7113056558110900224}]
I20250111 09:01:13.764734 15769 tablet_replica.cc:440] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P 0ad9dba2f090456ea13fa9cbea077682: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ee65a36c423b48658c01089b07a24c3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } } }
I20250111 09:01:13.764788 15764 tablet_replica.cc:440] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ee65a36c423b48658c01089b07a24c3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } } }
I20250111 09:01:13.764832 15769 tablet_replica.cc:443] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P 0ad9dba2f090456ea13fa9cbea077682: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:13.764878 15764 tablet_replica.cc:443] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P ee65a36c423b48658c01089b07a24c3d: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:13.767015 15723 tablet_replica.cc:440] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P c7db184c57854a4fadfb4502edf6f999: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ee65a36c423b48658c01089b07a24c3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7db184c57854a4fadfb4502edf6f999" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43947 } } peers { permanent_uuid: "ee65a36c423b48658c01089b07a24c3d" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45229 } } peers { permanent_uuid: "0ad9dba2f090456ea13fa9cbea077682" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 46065 } } }
I20250111 09:01:13.767100 15723 tablet_replica.cc:443] T 7044bb69c6e342ceb77cbcfe8b92c4b4 P c7db184c57854a4fadfb4502edf6f999: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:13.767508 15757 mvcc.cc:204] Tried to move back new op lower bound from 7113056558110900224 to 7113056557888303104. Current Snapshot: MvccSnapshot[applied={T|T < 7113056558110900224}]
I20250111 09:01:13.767513 15761 mvcc.cc:204] Tried to move back new op lower bound from 7113056558110900224 to 7113056557888303104. Current Snapshot: MvccSnapshot[applied={T|T < 7113056558110900224}]
I20250111 09:01:13.798537 15740 txn_write_ops-itest.cc:453] Txn 0 wrote row 13: OK
I20250111 09:01:13.825639 15740 txn_write_ops-itest.cc:453] Txn 0 wrote row 9: OK
W20250111 09:01:14.815655 15757 lock_manager.cc:494] Txn 1 has not acquired the partition lock after 1000ms
W20250111 09:01:14.816064 15757 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40144: Aborted: Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID
I20250111 09:01:14.816133 15793 ts_tablet_manager.cc:2109] Sending abort request for transaction 1
I20250111 09:01:14.816299 15757 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:40144 (ReqId={client: 0e9ea301c1d645b4b045e258c1d90504, seq_no=2, attempt_no=0}) took 1008 ms. Trace:
I20250111 09:01:14.816346 15757 rpcz_store.cc:276] 0111 09:01:13.807915 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:13.808090 (+ 175us) service_pool.cc:225] Handling call
0111 09:01:14.816290 (+1008200us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:13.812499 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:13.812538 (+ 39us) participant_op.cc:199] PREPARE: finished
0111 09:01:13.812560 (+ 22us) participant_op.cc:212] START. Timestamp: 1736586073812528
0111 09:01:13.812564 (+ 4us) op_driver.cc:348] REPLICATION: starting
0111 09:01:13.812649 (+ 85us) log.cc:844] Serialized 65 byte log entry
0111 09:01:13.814111 (+ 1462us) op_driver.cc:464] REPLICATION: finished
0111 09:01:13.814455 (+ 344us) participant_op.cc:265] APPLY: starting
0111 09:01:13.814614 (+ 159us) participant_op.cc:270] APPLY: finished
0111 09:01:13.814635 (+ 21us) log.cc:844] Serialized 14 byte log entry
0111 09:01:13.814656 (+ 21us) participant_op.cc:94] Released txn lock
0111 09:01:13.814661 (+ 5us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:01:13.815061 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 41b32a69149146839fe96ce9b5e0f246
0111 09:01:13.815108 (+ 47us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:13.815109 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:01:13.815110 (+ 1us) tablet.cc:659] Decoding operations
0111 09:01:13.815145 (+ 35us) write_op.cc:620] Acquiring the partition lock for write op
0111 09:01:14.815869 (+1000724us) op_driver.cc:389] HandleFailure(Aborted: Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID)
0111 09:01:14.815894 (+ 25us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:14.815910 (+ 16us) write_op.cc:454] Released schema lock
0111 09:01:14.815922 (+ 12us) write_op.cc:335] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":41,"txn-participant-registration.run_cpu_time_us":376,"txn-participant-registration.run_wall_time_us":4073,"child_traces":[["op",{"apply.queue_time_us":305,"apply.run_cpu_time_us":338,"apply.run_wall_time_us":338,"prepare.queue_time_us":57,"prepare.run_cpu_time_us":233,"prepare.run_wall_time_us":233,"raft.queue_time_us":102,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":2381,"replication_time_us":1529,"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":291,"thread_start_us":58,"threads_started":1,"txn-participant-registration.queue_time_us":136}]]}]]}
W20250111 09:01:14.817376 15708 batcher.cc:441] Aborted: Failed to write batch of 1 ops to tablet 41b32a69149146839fe96ce9b5e0f246 after 1 attempt(s): Failed to write to server: c7db184c57854a4fadfb4502edf6f999 (127.14.61.1:43947): Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID
I20250111 09:01:14.817741 15742 txn_write_ops-itest.cc:453] Txn 1 wrote row 3: IO error: failed to flush data: error details are available via KuduSession::GetPendingErrors()
I20250111 09:01:14.817812 15742 txn_write_ops-itest.cc:464] Txn 1 wrote row 3: Aborted: Failed to write batch of 1 ops to tablet 41b32a69149146839fe96ce9b5e0f246 after 1 attempt(s): Failed to write to server: c7db184c57854a4fadfb4502edf6f999 (127.14.61.1:43947): Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID
W20250111 09:01:14.821959 15757 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations
W20250111 09:01:15.017493 15761 lock_manager.cc:494] Txn 2 has not acquired the partition lock after 1000ms
W20250111 09:01:15.017786 15761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:37902: Aborted: Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID
I20250111 09:01:15.017843 15821 ts_tablet_manager.cc:2109] Sending abort request for transaction 2
I20250111 09:01:15.018085 15761 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:37902 (ReqId={client: 0e9ea301c1d645b4b045e258c1d90504, seq_no=3, attempt_no=0}) took 1007 ms. Trace:
I20250111 09:01:15.018152 15761 rpcz_store.cc:276] 0111 09:01:14.011001 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:14.011050 (+ 49us) service_pool.cc:225] Handling call
0111 09:01:15.018073 (+1007023us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:14.014955 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:14.014981 (+ 26us) participant_op.cc:199] PREPARE: finished
0111 09:01:14.014995 (+ 14us) participant_op.cc:212] START. Timestamp: 1736586074014930
0111 09:01:14.014996 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:01:14.015055 (+ 59us) log.cc:844] Serialized 65 byte log entry
0111 09:01:14.016493 (+ 1438us) op_driver.cc:464] REPLICATION: finished
0111 09:01:14.016583 (+ 90us) participant_op.cc:265] APPLY: starting
0111 09:01:14.016749 (+ 166us) participant_op.cc:270] APPLY: finished
0111 09:01:14.016774 (+ 25us) log.cc:844] Serialized 14 byte log entry
0111 09:01:14.016806 (+ 32us) participant_op.cc:94] Released txn lock
0111 09:01:14.016812 (+ 6us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:01:14.016965 (+ 0us) write_op.cc:183] PREPARE: starting on tablet f85169e5db994f3690e7cdd2ff16c75a
0111 09:01:14.017045 (+ 80us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:14.017046 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:01:14.017047 (+ 1us) tablet.cc:659] Decoding operations
0111 09:01:14.017082 (+ 35us) write_op.cc:620] Acquiring the partition lock for write op
0111 09:01:15.017631 (+1000549us) op_driver.cc:389] HandleFailure(Aborted: Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID)
0111 09:01:15.017644 (+ 13us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:15.017648 (+ 4us) write_op.cc:454] Released schema lock
0111 09:01:15.017653 (+ 5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":43,"txn-participant-registration.run_cpu_time_us":444,"txn-participant-registration.run_wall_time_us":3329,"child_traces":[["op",{"apply.queue_time_us":43,"apply.run_cpu_time_us":395,"apply.run_wall_time_us":395,"prepare.queue_time_us":358,"prepare.run_cpu_time_us":160,"prepare.run_wall_time_us":161,"raft.queue_time_us":93,"raft.run_cpu_time_us":352,"raft.run_wall_time_us":352,"replication_time_us":1478,"spinlock_wait_cycles":47616,"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":23,"thread_start_us":68,"threads_started":1,"txn-participant-registration.queue_time_us":140}]]}]]}
I20250111 09:01:15.019075 15739 txn_write_ops-itest.cc:453] Txn 2 wrote row 5: IO error: failed to flush data: error details are available via KuduSession::GetPendingErrors()
I20250111 09:01:15.019138 15739 txn_write_ops-itest.cc:464] Txn 2 wrote row 5: Aborted: Failed to write batch of 1 ops to tablet f85169e5db994f3690e7cdd2ff16c75a after 1 attempt(s): Failed to write to server: 0ad9dba2f090456ea13fa9cbea077682 (127.14.61.3:46065): Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID
I20250111 09:01:15.027494 15741 txn_write_ops-itest.cc:453] Txn 3 wrote row 6: OK
W20250111 09:01:15.824445 15757 lock_manager.cc:494] Txn 5 has not acquired the partition lock after 1000ms
W20250111 09:01:15.824795 15757 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40144: Aborted: Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID
I20250111 09:01:15.824885 15827 ts_tablet_manager.cc:2109] Sending abort request for transaction 5
I20250111 09:01:15.825049 15757 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:40144 (ReqId={client: 0e9ea301c1d645b4b045e258c1d90504, seq_no=6, attempt_no=0}) took 1738 ms. Trace:
I20250111 09:01:15.825119 15757 rpcz_store.cc:276] 0111 09:01:14.086154 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:14.086224 (+ 70us) service_pool.cc:225] Handling call
0111 09:01:15.825032 (+1738808us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:14.817896 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:14.817913 (+ 17us) participant_op.cc:199] PREPARE: finished
0111 09:01:14.817927 (+ 14us) participant_op.cc:212] START. Timestamp: 1736586074817907
0111 09:01:14.817928 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:01:14.817997 (+ 69us) log.cc:844] Serialized 65 byte log entry
0111 09:01:14.820303 (+ 2306us) op_driver.cc:464] REPLICATION: finished
0111 09:01:14.823303 (+ 3000us) participant_op.cc:265] APPLY: starting
0111 09:01:14.823452 (+ 149us) participant_op.cc:270] APPLY: finished
0111 09:01:14.823471 (+ 19us) log.cc:844] Serialized 14 byte log entry
0111 09:01:14.823753 (+ 282us) participant_op.cc:94] Released txn lock
0111 09:01:14.823757 (+ 4us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:01:14.823933 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 41b32a69149146839fe96ce9b5e0f246
0111 09:01:14.823999 (+ 66us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:14.824000 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:01:14.824002 (+ 2us) tablet.cc:659] Decoding operations
0111 09:01:14.824049 (+ 47us) write_op.cc:620] Acquiring the partition lock for write op
0111 09:01:15.824607 (+1000558us) op_driver.cc:389] HandleFailure(Aborted: Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID)
0111 09:01:15.824622 (+ 15us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:15.824628 (+ 6us) write_op.cc:454] Released schema lock
0111 09:01:15.824636 (+ 8us) write_op.cc:335] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":42,"txn-participant-registration.run_cpu_time_us":324,"txn-participant-registration.run_wall_time_us":3068,"child_traces":[["op",{"apply.queue_time_us":2959,"apply.run_cpu_time_us":371,"apply.run_wall_time_us":621,"prepare.queue_time_us":728365,"prepare.run_cpu_time_us":269,"prepare.run_wall_time_us":269,"raft.queue_time_us":657,"raft.run_cpu_time_us":7,"raft.run_wall_time_us":8,"replication_time_us":2360,"spinlock_wait_cycles":17152,"thread_start_us":173,"threads_started":3,"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":40,"thread_start_us":86,"threads_started":1,"txn-participant-registration.queue_time_us":191}]]}]]}
W20250111 09:01:15.825733 15708 batcher.cc:441] Aborted: Failed to write batch of 1 ops to tablet 41b32a69149146839fe96ce9b5e0f246 after 1 attempt(s): Failed to write to server: c7db184c57854a4fadfb4502edf6f999 (127.14.61.1:43947): Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID
I20250111 09:01:15.825930 15738 txn_write_ops-itest.cc:453] Txn 5 wrote row 2: IO error: failed to flush data: error details are available via KuduSession::GetPendingErrors()
I20250111 09:01:15.826000 15738 txn_write_ops-itest.cc:464] Txn 5 wrote row 2: Aborted: Failed to write batch of 1 ops to tablet 41b32a69149146839fe96ce9b5e0f246 after 1 attempt(s): Failed to write to server: c7db184c57854a4fadfb4502edf6f999 (127.14.61.1:43947): Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID
W20250111 09:01:16.025563 15761 lock_manager.cc:494] Txn 6 has not acquired the partition lock after 1000ms
W20250111 09:01:16.025825 15761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:37902: Aborted: Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID
I20250111 09:01:16.025916 15836 ts_tablet_manager.cc:2109] Sending abort request for transaction 6
I20250111 09:01:16.026012 15761 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:37902 (ReqId={client: 0e9ea301c1d645b4b045e258c1d90504, seq_no=7, attempt_no=0}) took 1828 ms. Trace:
I20250111 09:01:16.026074 15761 rpcz_store.cc:276] 0111 09:01:14.197791 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:14.197838 (+ 47us) service_pool.cc:225] Handling call
0111 09:01:16.026002 (+1828164us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:15.019347 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:15.019369 (+ 22us) participant_op.cc:199] PREPARE: finished
0111 09:01:15.019387 (+ 18us) participant_op.cc:212] START. Timestamp: 1736586075019321
0111 09:01:15.019388 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:01:15.019457 (+ 69us) log.cc:844] Serialized 65 byte log entry
0111 09:01:15.021757 (+ 2300us) op_driver.cc:464] REPLICATION: finished
0111 09:01:15.022093 (+ 336us) participant_op.cc:265] APPLY: starting
0111 09:01:15.022259 (+ 166us) participant_op.cc:270] APPLY: finished
0111 09:01:15.022285 (+ 26us) log.cc:844] Serialized 14 byte log entry
0111 09:01:15.022314 (+ 29us) participant_op.cc:94] Released txn lock
0111 09:01:15.022318 (+ 4us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:01:15.024907 (+ 0us) write_op.cc:183] PREPARE: starting on tablet f85169e5db994f3690e7cdd2ff16c75a
0111 09:01:15.024970 (+ 63us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:15.024972 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:01:15.024973 (+ 1us) tablet.cc:659] Decoding operations
0111 09:01:15.025035 (+ 62us) write_op.cc:620] Acquiring the partition lock for write op
0111 09:01:16.025667 (+1000632us) op_driver.cc:389] HandleFailure(Aborted: Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID)
0111 09:01:16.025678 (+ 11us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:16.025684 (+ 6us) write_op.cc:454] Released schema lock
0111 09:01:16.025692 (+ 8us) write_op.cc:335] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":37,"txn-participant-registration.run_cpu_time_us":340,"txn-participant-registration.run_wall_time_us":3221,"child_traces":[["op",{"apply.queue_time_us":300,"apply.run_cpu_time_us":392,"apply.run_wall_time_us":393,"prepare.queue_time_us":818082,"prepare.run_cpu_time_us":169,"prepare.run_wall_time_us":168,"raft.queue_time_us":119,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":3,"replication_time_us":2358,"spinlock_wait_cycles":54656,"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":2448,"thread_start_us":71,"threads_started":1,"txn-participant-registration.queue_time_us":164}]]}]]}
W20250111 09:01:16.026630 15761 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations
I20250111 09:01:16.026818 15736 txn_write_ops-itest.cc:453] Txn 6 wrote row 10: IO error: failed to flush data: error details are available via KuduSession::GetPendingErrors()
I20250111 09:01:16.026899 15736 txn_write_ops-itest.cc:464] Txn 6 wrote row 10: Aborted: Failed to write batch of 1 ops to tablet f85169e5db994f3690e7cdd2ff16c75a after 1 attempt(s): Failed to write to server: 0ad9dba2f090456ea13fa9cbea077682 (127.14.61.3:46065): Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID
I20250111 09:01:16.028954 15835 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.ParticipateInTransaction from 127.14.61.2:59397 (ReqId={client: 23f2e7e8432443b1b3d0a0c27dbf833b, seq_no=5, attempt_no=0}) took 1006 ms. Trace:
I20250111 09:01:16.029199 15835 rpcz_store.cc:276] 0111 09:01:15.022603 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:15.022652 (+ 49us) service_pool.cc:225] Handling call
0111 09:01:16.028931 (+1006279us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:16.026590 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:16.026770 (+ 180us) participant_op.cc:199] PREPARE: finished
0111 09:01:16.026798 (+ 28us) participant_op.cc:212] START. Timestamp: 1736586076026731
0111 09:01:16.026800 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:01:16.026919 (+ 119us) log.cc:844] Serialized 65 byte log entry
0111 09:01:16.028697 (+ 1778us) op_driver.cc:464] REPLICATION: finished
0111 09:01:16.028763 (+ 66us) participant_op.cc:265] APPLY: starting
0111 09:01:16.028852 (+ 89us) participant_op.cc:270] APPLY: finished
0111 09:01:16.028886 (+ 34us) log.cc:844] Serialized 14 byte log entry
0111 09:01:16.028910 (+ 24us) participant_op.cc:94] Released txn lock
0111 09:01:16.028918 (+ 8us) participant_op.cc:291] FINISH: Op applied
Metrics: {"child_traces":[["op",{"apply.queue_time_us":36,"prepare.queue_time_us":1003799,"prepare.run_cpu_time_us":466,"prepare.run_wall_time_us":466,"raft.queue_time_us":133,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":422,"replication_time_us":1879,"thread_start_us":61,"threads_started":1}]]}
W20250111 09:01:16.826247 15757 lock_manager.cc:494] Txn 4 has not acquired the partition lock after 1000ms
W20250111 09:01:16.826514 15757 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40144: Aborted: Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID
I20250111 09:01:16.826648 15842 ts_tablet_manager.cc:2109] Sending abort request for transaction 4
I20250111 09:01:16.826651 15757 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:40144 (ReqId={client: 0e9ea301c1d645b4b045e258c1d90504, seq_no=5, attempt_no=0}) took 2786 ms. Trace:
I20250111 09:01:16.826738 15757 rpcz_store.cc:276] 0111 09:01:14.040353 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:14.040397 (+ 44us) service_pool.cc:225] Handling call
0111 09:01:16.826640 (+2786243us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:14.817562 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:14.817595 (+ 33us) participant_op.cc:199] PREPARE: finished
0111 09:01:14.817623 (+ 28us) participant_op.cc:212] START. Timestamp: 1736586074817601
0111 09:01:14.817624 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:01:14.817700 (+ 76us) log.cc:844] Serialized 65 byte log entry
0111 09:01:14.820199 (+ 2499us) op_driver.cc:464] REPLICATION: finished
0111 09:01:14.822105 (+ 1906us) participant_op.cc:265] APPLY: starting
0111 09:01:14.824135 (+ 2030us) participant_op.cc:270] APPLY: finished
0111 09:01:14.824154 (+ 19us) log.cc:844] Serialized 14 byte log entry
0111 09:01:14.824180 (+ 26us) participant_op.cc:94] Released txn lock
0111 09:01:14.824185 (+ 5us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:01:15.825651 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 41b32a69149146839fe96ce9b5e0f246
0111 09:01:15.825778 (+ 127us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:15.825780 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:01:15.825782 (+ 2us) tablet.cc:659] Decoding operations
0111 09:01:15.825863 (+ 81us) write_op.cc:620] Acquiring the partition lock for write op
0111 09:01:16.826357 (+1000494us) op_driver.cc:389] HandleFailure(Aborted: Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID)
0111 09:01:16.826367 (+ 10us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:16.826372 (+ 5us) write_op.cc:454] Released schema lock
0111 09:01:16.826377 (+ 5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":39,"txn-participant-registration.run_cpu_time_us":320,"txn-participant-registration.run_wall_time_us":3482,"child_traces":[["op",{"apply.queue_time_us":1877,"apply.run_cpu_time_us":334,"apply.run_wall_time_us":2212,"mutex_wait_us":82,"prepare.queue_time_us":773453,"prepare.run_cpu_time_us":308,"prepare.run_wall_time_us":308,"raft.queue_time_us":1211,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":5,"replication_time_us":2563,"thread_start_us":177,"threads_started":3,"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":1001332,"thread_start_us":79,"threads_started":1,"txn-participant-registration.queue_time_us":201}]]}]]}
W20250111 09:01:16.827306 15708 batcher.cc:441] Aborted: Failed to write batch of 1 ops to tablet 41b32a69149146839fe96ce9b5e0f246 after 1 attempt(s): Failed to write to server: c7db184c57854a4fadfb4502edf6f999 (127.14.61.1:43947): Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID
I20250111 09:01:16.827462 15743 txn_write_ops-itest.cc:453] Txn 4 wrote row 7: IO error: failed to flush data: error details are available via KuduSession::GetPendingErrors()
I20250111 09:01:16.827518 15743 txn_write_ops-itest.cc:464] Txn 4 wrote row 7: Aborted: Failed to write batch of 1 ops to tablet 41b32a69149146839fe96ce9b5e0f246 after 1 attempt(s): Failed to write to server: c7db184c57854a4fadfb4502edf6f999 (127.14.61.1:43947): Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID
W20250111 09:01:16.827662 15757 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations
I20250111 09:01:16.830682 15850 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.ParticipateInTransaction from 127.14.61.2:43247 (ReqId={client: 23f2e7e8432443b1b3d0a0c27dbf833b, seq_no=4, attempt_no=0}) took 2003 ms. Trace:
I20250111 09:01:16.830792 15850 rpcz_store.cc:276] 0111 09:01:14.827440 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:14.827508 (+ 68us) service_pool.cc:225] Handling call
0111 09:01:16.830652 (+2003144us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:16.827200 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:16.827260 (+ 60us) participant_op.cc:199] PREPARE: finished
0111 09:01:16.827271 (+ 11us) participant_op.cc:212] START. Timestamp: 1736586076827246
0111 09:01:16.827272 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:01:16.827389 (+ 117us) log.cc:844] Serialized 75 byte log entry
0111 09:01:16.829594 (+ 2205us) op_driver.cc:464] REPLICATION: finished
0111 09:01:16.830501 (+ 907us) participant_op.cc:265] APPLY: starting
0111 09:01:16.830579 (+ 78us) participant_op.cc:270] APPLY: finished
0111 09:01:16.830607 (+ 28us) log.cc:844] Serialized 14 byte log entry
0111 09:01:16.830630 (+ 23us) participant_op.cc:94] Released txn lock
0111 09:01:16.830639 (+ 9us) participant_op.cc:291] FINISH: Op applied
Metrics: {"child_traces":[["op",{"apply.queue_time_us":867,"prepare.queue_time_us":1999549,"prepare.run_cpu_time_us":411,"prepare.run_wall_time_us":411,"raft.queue_time_us":192,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":294,"replication_time_us":2304,"thread_start_us":185,"threads_started":3,"wal-append.queue_time_us":484}]]}
I20250111 09:01:16.833495 15855 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.ParticipateInTransaction from 127.14.61.2:43247 (ReqId={client: 23f2e7e8432443b1b3d0a0c27dbf833b, seq_no=6, attempt_no=0}) took 1003 ms. Trace:
I20250111 09:01:16.833586 15855 rpcz_store.cc:276] 0111 09:01:15.829704 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:15.829741 (+ 37us) service_pool.cc:225] Handling call
0111 09:01:16.833475 (+1003734us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:16.827629 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:16.827728 (+ 99us) participant_op.cc:199] PREPARE: finished
0111 09:01:16.827748 (+ 20us) participant_op.cc:212] START. Timestamp: 1736586076827721
0111 09:01:16.827750 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:01:16.827820 (+ 70us) log.cc:844] Serialized 65 byte log entry
0111 09:01:16.831989 (+ 4169us) op_driver.cc:464] REPLICATION: finished
0111 09:01:16.833197 (+ 1208us) participant_op.cc:265] APPLY: starting
0111 09:01:16.833271 (+ 74us) participant_op.cc:270] APPLY: finished
0111 09:01:16.833295 (+ 24us) log.cc:844] Serialized 14 byte log entry
0111 09:01:16.833451 (+ 156us) participant_op.cc:94] Released txn lock
0111 09:01:16.833461 (+ 10us) participant_op.cc:291] FINISH: Op applied
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1178,"prepare.queue_time_us":997746,"prepare.run_cpu_time_us":216,"prepare.run_wall_time_us":216,"replication_time_us":4221,"thread_start_us":94,"threads_started":1}]]}
W20250111 09:01:17.027786 15761 lock_manager.cc:494] Txn 7 has not acquired the partition lock after 1000ms
W20250111 09:01:17.028057 15761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:37902: Aborted: Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID
I20250111 09:01:17.028162 15863 ts_tablet_manager.cc:2109] Sending abort request for transaction 7
I20250111 09:01:17.028191 15761 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:37902 (ReqId={client: 0e9ea301c1d645b4b045e258c1d90504, seq_no=8, attempt_no=0}) took 2764 ms. Trace:
I20250111 09:01:17.028265 15761 rpcz_store.cc:276] 0111 09:01:14.264130 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:14.264187 (+ 57us) service_pool.cc:225] Handling call
0111 09:01:17.028182 (+2763995us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:15.019531 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:15.019546 (+ 15us) participant_op.cc:199] PREPARE: finished
0111 09:01:15.019558 (+ 12us) participant_op.cc:212] START. Timestamp: 1736586075019496
0111 09:01:15.019560 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:01:15.019625 (+ 65us) log.cc:844] Serialized 65 byte log entry
0111 09:01:15.024055 (+ 4430us) op_driver.cc:464] REPLICATION: finished
0111 09:01:15.024100 (+ 45us) participant_op.cc:265] APPLY: starting
0111 09:01:15.024255 (+ 155us) participant_op.cc:270] APPLY: finished
0111 09:01:15.024273 (+ 18us) log.cc:844] Serialized 14 byte log entry
0111 09:01:15.024295 (+ 22us) participant_op.cc:94] Released txn lock
0111 09:01:15.024298 (+ 3us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:01:16.027086 (+ 0us) write_op.cc:183] PREPARE: starting on tablet f85169e5db994f3690e7cdd2ff16c75a
0111 09:01:16.027168 (+ 82us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:16.027170 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:01:16.027171 (+ 1us) tablet.cc:659] Decoding operations
0111 09:01:16.027240 (+ 69us) write_op.cc:620] Acquiring the partition lock for write op
0111 09:01:17.027894 (+1000654us) op_driver.cc:389] HandleFailure(Aborted: Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID)
0111 09:01:17.027904 (+ 10us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:17.027908 (+ 4us) write_op.cc:454] Released schema lock
0111 09:01:17.027913 (+ 5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":40,"txn-participant-registration.run_cpu_time_us":331,"txn-participant-registration.run_wall_time_us":3328,"child_traces":[["op",{"apply.queue_time_us":16,"apply.run_cpu_time_us":299,"apply.run_wall_time_us":311,"mutex_wait_us":2,"prepare.queue_time_us":751813,"prepare.run_cpu_time_us":149,"prepare.run_wall_time_us":149,"replication_time_us":4478,"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":1002690,"thread_start_us":101,"threads_started":1,"txn-participant-registration.queue_time_us":191}]]}]]}
I20250111 09:01:17.028932 15737 txn_write_ops-itest.cc:453] Txn 7 wrote row 14: IO error: failed to flush data: error details are available via KuduSession::GetPendingErrors()
I20250111 09:01:17.029016 15737 txn_write_ops-itest.cc:464] Txn 7 wrote row 14: Aborted: Failed to write batch of 1 ops to tablet f85169e5db994f3690e7cdd2ff16c75a after 1 attempt(s): Failed to write to server: 0ad9dba2f090456ea13fa9cbea077682 (127.14.61.3:46065): Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID
W20250111 09:01:17.029325 15761 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations
I20250111 09:01:17.031805 15854 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:37902 (ReqId={client: 0e9ea301c1d645b4b045e258c1d90504, seq_no=9, attempt_no=0}) took 2003 ms. Trace:
I20250111 09:01:17.033291 15853 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.ParticipateInTransaction from 127.14.61.2:59397 (ReqId={client: 23f2e7e8432443b1b3d0a0c27dbf833b, seq_no=7, attempt_no=0}) took 1002 ms. Trace:
I20250111 09:01:17.033371 15853 rpcz_store.cc:276] 0111 09:01:16.030458 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:16.030499 (+ 41us) service_pool.cc:225] Handling call
0111 09:01:17.033271 (+1002772us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:17.029299 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:17.029384 (+ 85us) participant_op.cc:199] PREPARE: finished
0111 09:01:17.029406 (+ 22us) participant_op.cc:212] START. Timestamp: 1736586077029341
0111 09:01:17.029409 (+ 3us) op_driver.cc:348] REPLICATION: starting
0111 09:01:17.029503 (+ 94us) log.cc:844] Serialized 65 byte log entry
0111 09:01:17.033027 (+ 3524us) op_driver.cc:464] REPLICATION: finished
0111 09:01:17.033097 (+ 70us) participant_op.cc:265] APPLY: starting
0111 09:01:17.033197 (+ 100us) participant_op.cc:270] APPLY: finished
0111 09:01:17.033224 (+ 27us) log.cc:844] Serialized 14 byte log entry
0111 09:01:17.033248 (+ 24us) participant_op.cc:94] Released txn lock
0111 09:01:17.033257 (+ 9us) participant_op.cc:291] FINISH: Op applied
Metrics: {"child_traces":[["op",{"apply.queue_time_us":39,"prepare.queue_time_us":998655,"prepare.run_cpu_time_us":235,"prepare.run_wall_time_us":234,"replication_time_us":3576,"spinlock_wait_cycles":10368}]]}
I20250111 09:01:17.034113 15854 rpcz_store.cc:276] 0111 09:01:15.028084 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:15.028127 (+ 43us) service_pool.cc:225] Handling call
0111 09:01:17.031793 (+2003666us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:17.028690 (+ 0us) write_op.cc:183] PREPARE: starting on tablet f85169e5db994f3690e7cdd2ff16c75a
0111 09:01:17.028775 (+ 85us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:17.028776 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:01:17.028778 (+ 2us) tablet.cc:659] Decoding operations
0111 09:01:17.028834 (+ 56us) write_op.cc:620] Acquiring the partition lock for write op
0111 09:01:17.028839 (+ 5us) write_op.cc:641] Partition lock acquired for write op
0111 09:01:17.028841 (+ 2us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:01:17.028872 (+ 31us) tablet.cc:698] Row locks acquired
0111 09:01:17.028873 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:01:17.028916 (+ 43us) write_op.cc:270] Start()
0111 09:01:17.028942 (+ 26us) write_op.cc:276] Timestamp: P: 1736586077028854 usec, L: 0
0111 09:01:17.028945 (+ 3us) op_driver.cc:348] REPLICATION: starting
0111 09:01:17.029114 (+ 169us) log.cc:844] Serialized 302 byte log entry
0111 09:01:17.031189 (+ 2075us) op_driver.cc:464] REPLICATION: finished
0111 09:01:17.031366 (+ 177us) write_op.cc:301] APPLY: starting
0111 09:01:17.031384 (+ 18us) tablet.cc:1364] starting BulkCheckPresence
0111 09:01:17.031435 (+ 51us) tablet.cc:1367] finished BulkCheckPresence
0111 09:01:17.031446 (+ 11us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:01:17.031500 (+ 54us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:01:17.031524 (+ 24us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0111 09:01:17.031530 (+ 6us) write_op.cc:312] APPLY: finished
0111 09:01:17.031588 (+ 58us) log.cc:844] Serialized 24 byte log entry
0111 09:01:17.031721 (+ 133us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:17.031735 (+ 14us) write_op.cc:454] Released schema lock
0111 09:01:17.031739 (+ 4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":138,"num_ops":1,"prepare.queue_time_us":2000322,"prepare.run_cpu_time_us":596,"prepare.run_wall_time_us":596,"raft.queue_time_us":274,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":309,"replication_time_us":2222,"thread_start_us":80,"threads_started":1}]]}
I20250111 09:01:17.034838 15741 txn_write_ops-itest.cc:453] Txn 3 wrote row 1: OK
I20250111 09:01:19.832477 15742 txn_write_ops-itest.cc:453] Txn 8 wrote row 3: OK
I20250111 09:01:19.841220 15742 txn_write_ops-itest.cc:453] Txn 8 wrote row 11: OK
I20250111 09:01:20.033699 15739 txn_write_ops-itest.cc:453] Txn 9 wrote row 5: OK
I20250111 09:01:20.043107 15739 txn_write_ops-itest.cc:453] Txn 9 wrote row 15: OK
I20250111 09:01:20.840641 15738 txn_write_ops-itest.cc:453] Txn 10 wrote row 2: OK
I20250111 09:01:20.849926 15738 txn_write_ops-itest.cc:453] Txn 10 wrote row 12: OK
I20250111 09:01:21.040896 15736 txn_write_ops-itest.cc:453] Txn 11 wrote row 10: OK
I20250111 09:01:21.043893 15736 txn_write_ops-itest.cc:453] Txn 11 wrote row 4: OK
I20250111 09:01:21.843005 15743 txn_write_ops-itest.cc:453] Txn 12 wrote row 7: OK
I20250111 09:01:21.846336 15743 txn_write_ops-itest.cc:453] Txn 12 wrote row 16: OK
I20250111 09:01:22.043072 15737 txn_write_ops-itest.cc:453] Txn 13 wrote row 14: OK
I20250111 09:01:22.051899 15737 txn_write_ops-itest.cc:453] Txn 13 wrote row 8: OK
I20250111 09:01:22.079233 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 15295
I20250111 09:01:22.085211 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 15432
I20250111 09:01:22.094290 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 15569
I20250111 09:01:22.100000 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 15189
2025-01-11T09:01:22Z chronyd exiting
[ OK ] TxnWriteOpsITest.DeadlockPrevention (9676 ms)
[ RUN ] TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds
2025-01-11T09:01:22Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-01-11T09:01:22Z Disabled control of system clock
I20250111 09:01:22.128438 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.61.62:46109
--webserver_interface=127.14.61.62
--webserver_port=0
--builtin_ntp_servers=127.14.61.20:37853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.61.62:46109
--txn_manager_enabled=true
--txn_manager_status_table_range_partition_span=3 with env {}
W20250111 09:01:22.221210 15940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:01:22.221379 15940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:01:22.221452 15940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:01:22.224242 15940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250111 09:01:22.224283 15940 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20250111 09:01:22.224298 15940 flags.cc:425] Enabled experimental flag: --txn_manager_status_table_range_partition_span=3
W20250111 09:01:22.224315 15940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:01:22.224331 15940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250111 09:01:22.224350 15940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250111 09:01:22.227667 15940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:37853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.61.62:46109
--txn_manager_enabled=true
--txn_manager_status_table_range_partition_span=3
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.61.62:46109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.61.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.15940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:01:22.228375 15940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:01:22.229238 15940 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250111 09:01:22.235710 15940 server_base.cc:1034] running on GCE node
W20250111 09:01:22.235898 15948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:22.236517 15946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:22.237061 15945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:01:22.237432 15940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:01:22.238143 15940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:01:22.239280 15940 hybrid_clock.cc:648] HybridClock initialized: now 1736586082239252 us; error 40 us; skew 500 ppm
I20250111 09:01:22.240826 15940 webserver.cc:458] Webserver started at http://127.14.61.62:33427/ using document root <none> and password file <none>
I20250111 09:01:22.241299 15940 fs_manager.cc:362] Metadata directory not provided
I20250111 09:01:22.241345 15940 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:01:22.241492 15940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:01:22.242873 15940 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/master-0/data/instance:
uuid: "556a1f23f24d4b5ba7f0f381d156ced6"
format_stamp: "Formatted at 2025-01-11 09:01:22 on dist-test-slave-nvbt"
I20250111 09:01:22.243285 15940 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/master-0/wal/instance:
uuid: "556a1f23f24d4b5ba7f0f381d156ced6"
format_stamp: "Formatted at 2025-01-11 09:01:22 on dist-test-slave-nvbt"
I20250111 09:01:22.246206 15940 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20250111 09:01:22.247992 15954 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:22.248819 15940 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:01:22.248907 15940 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/master-0/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/master-0/wal
uuid: "556a1f23f24d4b5ba7f0f381d156ced6"
format_stamp: "Formatted at 2025-01-11 09:01:22 on dist-test-slave-nvbt"
I20250111 09:01:22.248972 15940 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:01:22.272544 15940 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:01:22.273080 15940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:01:22.273217 15940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:01:22.279500 15940 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:46109
I20250111 09:01:22.279542 16016 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:46109 every 8 connection(s)
I20250111 09:01:22.280243 15940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
I20250111 09:01:22.282227 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 15940
I20250111 09:01:22.282250 16017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:22.282317 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/master-0/wal/instance
I20250111 09:01:22.287227 16017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6: Bootstrap starting.
I20250111 09:01:22.289276 16017 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:22.289881 16017 log.cc:826] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6: Log is configured to *not* fsync() on all Append() calls
I20250111 09:01:22.291352 16017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6: No bootstrap required, opened a new log
I20250111 09:01:22.293951 16017 raft_consensus.cc:357] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "556a1f23f24d4b5ba7f0f381d156ced6" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 46109 } }
I20250111 09:01:22.294121 16017 raft_consensus.cc:383] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:22.294165 16017 raft_consensus.cc:738] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 556a1f23f24d4b5ba7f0f381d156ced6, State: Initialized, Role: FOLLOWER
I20250111 09:01:22.294489 16017 consensus_queue.cc:260] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "556a1f23f24d4b5ba7f0f381d156ced6" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 46109 } }
I20250111 09:01:22.294576 16017 raft_consensus.cc:397] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:01:22.294615 16017 raft_consensus.cc:491] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:01:22.294682 16017 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:22.295269 16017 raft_consensus.cc:513] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "556a1f23f24d4b5ba7f0f381d156ced6" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 46109 } }
I20250111 09:01:22.295526 16017 leader_election.cc:304] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 556a1f23f24d4b5ba7f0f381d156ced6; no voters:
I20250111 09:01:22.295719 16017 leader_election.cc:290] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:01:22.295809 16022 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:22.296000 16022 raft_consensus.cc:695] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6 [term 1 LEADER]: Becoming Leader. State: Replica: 556a1f23f24d4b5ba7f0f381d156ced6, State: Running, Role: LEADER
I20250111 09:01:22.296272 16022 consensus_queue.cc:237] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "556a1f23f24d4b5ba7f0f381d156ced6" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 46109 } }
I20250111 09:01:22.296713 16017 sys_catalog.cc:564] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6 [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:01:22.297215 16023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 556a1f23f24d4b5ba7f0f381d156ced6. Latest consensus state: current_term: 1 leader_uuid: "556a1f23f24d4b5ba7f0f381d156ced6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "556a1f23f24d4b5ba7f0f381d156ced6" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 46109 } } }
I20250111 09:01:22.297374 16023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6 [sys.catalog]: This master's current role is: LEADER
I20250111 09:01:22.297516 16024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "556a1f23f24d4b5ba7f0f381d156ced6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "556a1f23f24d4b5ba7f0f381d156ced6" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 46109 } } }
I20250111 09:01:22.297617 16024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6 [sys.catalog]: This master's current role is: LEADER
I20250111 09:01:22.297732 16029 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:01:22.300001 16029 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:01:22.304376 16029 catalog_manager.cc:1348] Generated new cluster ID: 8d1525aaecd642e7840039437709440a
I20250111 09:01:22.304426 16029 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:01:22.313241 16029 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:01:22.313982 16029 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:01:22.321938 16029 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6: Generated new TSK 0
I20250111 09:01:22.322431 16029 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:01:22.331233 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.1:0
--local_ip_for_outbound_sockets=127.14.61.1
--webserver_interface=127.14.61.1
--webserver_port=0
--tserver_master_addrs=127.14.61.62:46109
--builtin_ntp_servers=127.14.61.20:37853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_txn_system_client_init=true with env {}
W20250111 09:01:22.421854 16041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:01:22.422005 16041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:01:22.422042 16041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:01:22.424932 16041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:01:22.424996 16041 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:01:22.425084 16041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.1
I20250111 09:01:22.428151 16041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:37853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.61.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:46109
--never_fsync=true
--heap_profile_path=/tmp/kudu.16041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.1
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:01:22.428885 16041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:01:22.429760 16041 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:01:22.436561 16046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:22.436650 16049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:22.436853 16047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:01:22.437536 16041 server_base.cc:1034] running on GCE node
I20250111 09:01:22.437796 16041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:01:22.438220 16041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:01:22.439389 16041 hybrid_clock.cc:648] HybridClock initialized: now 1736586082439368 us; error 39 us; skew 500 ppm
I20250111 09:01:22.440831 16041 webserver.cc:458] Webserver started at http://127.14.61.1:36275/ using document root <none> and password file <none>
I20250111 09:01:22.441290 16041 fs_manager.cc:362] Metadata directory not provided
I20250111 09:01:22.441335 16041 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:01:22.441479 16041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:01:22.442811 16041 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-0/data/instance:
uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8"
format_stamp: "Formatted at 2025-01-11 09:01:22 on dist-test-slave-nvbt"
I20250111 09:01:22.443221 16041 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-0/wal/instance:
uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8"
format_stamp: "Formatted at 2025-01-11 09:01:22 on dist-test-slave-nvbt"
I20250111 09:01:22.446197 16041 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20250111 09:01:22.447990 16055 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:22.448763 16041 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:01:22.448843 16041 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-0/wal
uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8"
format_stamp: "Formatted at 2025-01-11 09:01:22 on dist-test-slave-nvbt"
I20250111 09:01:22.448915 16041 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:01:22.470101 16041 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:01:22.470504 16041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:01:22.470626 16041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:01:22.471779 16041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:01:22.471827 16041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:22.471871 16041 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:01:22.471904 16041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:22.481118 16041 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:35063
I20250111 09:01:22.481158 16175 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:35063 every 8 connection(s)
I20250111 09:01:22.481806 16041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
I20250111 09:01:22.482965 16176 heartbeater.cc:346] Connected to a master server at 127.14.61.62:46109
I20250111 09:01:22.483078 16176 heartbeater.cc:463] Registering TS with master...
I20250111 09:01:22.483469 16176 heartbeater.cc:510] Master 127.14.61.62:46109 requested a full tablet report, sending...
I20250111 09:01:22.484566 15971 ts_manager.cc:194] Registered new tserver with Master: 91e9fcf6b8ad407dbbf5c8339221b1b8 (127.14.61.1:35063)
I20250111 09:01:22.485749 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 16041
I20250111 09:01:22.485740 15971 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.1:43705
I20250111 09:01:22.485858 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-0/wal/instance
I20250111 09:01:22.487500 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.2:0
--local_ip_for_outbound_sockets=127.14.61.2
--webserver_interface=127.14.61.2
--webserver_port=0
--tserver_master_addrs=127.14.61.62:46109
--builtin_ntp_servers=127.14.61.20:37853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_txn_system_client_init=true with env {}
W20250111 09:01:22.577608 16178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:01:22.577771 16178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:01:22.577812 16178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:01:22.580492 16178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:01:22.580557 16178 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:01:22.580610 16178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.2
I20250111 09:01:22.583767 16178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:37853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.61.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:46109
--never_fsync=true
--heap_profile_path=/tmp/kudu.16178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.2
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:01:22.584535 16178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:01:22.585520 16178 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:01:22.592108 16184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:22.592123 16183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:22.592345 16186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:01:22.593063 16178 server_base.cc:1034] running on GCE node
I20250111 09:01:22.593343 16178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:01:22.593819 16178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:01:22.594965 16178 hybrid_clock.cc:648] HybridClock initialized: now 1736586082594942 us; error 36 us; skew 500 ppm
I20250111 09:01:22.596529 16178 webserver.cc:458] Webserver started at http://127.14.61.2:46565/ using document root <none> and password file <none>
I20250111 09:01:22.596997 16178 fs_manager.cc:362] Metadata directory not provided
I20250111 09:01:22.597098 16178 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:01:22.597298 16178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:01:22.598660 16178 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-1/data/instance:
uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad"
format_stamp: "Formatted at 2025-01-11 09:01:22 on dist-test-slave-nvbt"
I20250111 09:01:22.599067 16178 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-1/wal/instance:
uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad"
format_stamp: "Formatted at 2025-01-11 09:01:22 on dist-test-slave-nvbt"
I20250111 09:01:22.601691 16178 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20250111 09:01:22.603420 16192 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:22.604264 16178 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:01:22.604353 16178 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-1/wal
uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad"
format_stamp: "Formatted at 2025-01-11 09:01:22 on dist-test-slave-nvbt"
I20250111 09:01:22.604422 16178 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:01:22.614331 16178 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:01:22.614735 16178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:01:22.614862 16178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:01:22.615947 16178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:01:22.615998 16178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:22.616037 16178 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:01:22.616063 16178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:22.627256 16178 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.2:37009
I20250111 09:01:22.627295 16312 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.2:37009 every 8 connection(s)
I20250111 09:01:22.628158 16178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
I20250111 09:01:22.629215 16313 heartbeater.cc:346] Connected to a master server at 127.14.61.62:46109
I20250111 09:01:22.629323 16313 heartbeater.cc:463] Registering TS with master...
I20250111 09:01:22.629743 16313 heartbeater.cc:510] Master 127.14.61.62:46109 requested a full tablet report, sending...
I20250111 09:01:22.630487 15971 ts_manager.cc:194] Registered new tserver with Master: 11c4b3d95f0e4ff2b317b15ff32ff0ad (127.14.61.2:37009)
I20250111 09:01:22.631086 15971 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.2:54641
I20250111 09:01:22.632100 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 16178
I20250111 09:01:22.632211 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-1/wal/instance
I20250111 09:01:22.634193 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.3:0
--local_ip_for_outbound_sockets=127.14.61.3
--webserver_interface=127.14.61.3
--webserver_port=0
--tserver_master_addrs=127.14.61.62:46109
--builtin_ntp_servers=127.14.61.20:37853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_txn_system_client_init=true with env {}
W20250111 09:01:22.726864 16315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:01:22.727013 16315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:01:22.727051 16315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:01:22.729765 16315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:01:22.729835 16315 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:01:22.729907 16315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.3
I20250111 09:01:22.733008 16315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:37853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.61.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:46109
--never_fsync=true
--heap_profile_path=/tmp/kudu.16315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.3
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:01:22.733865 16315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:01:22.734830 16315 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:01:22.741495 16323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:22.742121 16321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:22.742383 16320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:01:22.743064 16315 server_base.cc:1034] running on GCE node
I20250111 09:01:22.743350 16315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:01:22.743839 16315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:01:22.745023 16315 hybrid_clock.cc:648] HybridClock initialized: now 1736586082744975 us; error 56 us; skew 500 ppm
I20250111 09:01:22.746614 16315 webserver.cc:458] Webserver started at http://127.14.61.3:45975/ using document root <none> and password file <none>
I20250111 09:01:22.747071 16315 fs_manager.cc:362] Metadata directory not provided
I20250111 09:01:22.747116 16315 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:01:22.747248 16315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:01:22.748566 16315 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-2/data/instance:
uuid: "2cb0b3b32eb94bf688d9587b86222a0a"
format_stamp: "Formatted at 2025-01-11 09:01:22 on dist-test-slave-nvbt"
I20250111 09:01:22.748945 16315 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-2/wal/instance:
uuid: "2cb0b3b32eb94bf688d9587b86222a0a"
format_stamp: "Formatted at 2025-01-11 09:01:22 on dist-test-slave-nvbt"
I20250111 09:01:22.751732 16315 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20250111 09:01:22.753609 16329 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:22.754463 16315 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:01:22.754549 16315 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-2/wal
uuid: "2cb0b3b32eb94bf688d9587b86222a0a"
format_stamp: "Formatted at 2025-01-11 09:01:22 on dist-test-slave-nvbt"
I20250111 09:01:22.754634 16315 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:01:22.769531 16315 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:01:22.769946 16315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:01:22.770069 16315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:01:22.771211 16315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:01:22.771257 16315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:22.771307 16315 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:01:22.771338 16315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:22.780622 16315 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.3:42055
I20250111 09:01:22.780663 16449 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.3:42055 every 8 connection(s)
I20250111 09:01:22.781458 16315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
I20250111 09:01:22.782476 16450 heartbeater.cc:346] Connected to a master server at 127.14.61.62:46109
I20250111 09:01:22.782882 16450 heartbeater.cc:463] Registering TS with master...
I20250111 09:01:22.783378 16450 heartbeater.cc:510] Master 127.14.61.62:46109 requested a full tablet report, sending...
I20250111 09:01:22.784189 15971 ts_manager.cc:194] Registered new tserver with Master: 2cb0b3b32eb94bf688d9587b86222a0a (127.14.61.3:42055)
I20250111 09:01:22.784827 15971 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.3:57237
I20250111 09:01:22.788506 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 16315
I20250111 09:01:22.788594 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds.1736586070622223-14580-0/minicluster-data/ts-2/wal/instance
I20250111 09:01:22.790606 14580 external_mini_cluster.cc:934] 3 TS(s) registered with all masters
I20250111 09:01:22.797853 15971 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33084:
name: "txn_write_ops_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
W20250111 09:01:22.800244 15971 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table txn_write_ops_test in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250111 09:01:22.814366 16104 tablet_service.cc:1467] Processing CreateTablet for tablet 322978c9a67a4ec4853b6bdaeda4a5dd (DEFAULT_TABLE table=txn_write_ops_test [id=701270f4515540f086c3f75b48e2fc66]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:22.814369 16105 tablet_service.cc:1467] Processing CreateTablet for tablet bf5911377e9c4596beb72781c5f09317 (DEFAULT_TABLE table=txn_write_ops_test [id=701270f4515540f086c3f75b48e2fc66]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:22.814672 16375 tablet_service.cc:1467] Processing CreateTablet for tablet 322978c9a67a4ec4853b6bdaeda4a5dd (DEFAULT_TABLE table=txn_write_ops_test [id=701270f4515540f086c3f75b48e2fc66]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:22.814682 16242 tablet_service.cc:1467] Processing CreateTablet for tablet bf5911377e9c4596beb72781c5f09317 (DEFAULT_TABLE table=txn_write_ops_test [id=701270f4515540f086c3f75b48e2fc66]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:22.815315 16104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 322978c9a67a4ec4853b6bdaeda4a5dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:22.815315 16375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 322978c9a67a4ec4853b6bdaeda4a5dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:22.815394 16242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf5911377e9c4596beb72781c5f09317. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:22.815589 16105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf5911377e9c4596beb72781c5f09317. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:22.816756 16373 tablet_service.cc:1467] Processing CreateTablet for tablet bf5911377e9c4596beb72781c5f09317 (DEFAULT_TABLE table=txn_write_ops_test [id=701270f4515540f086c3f75b48e2fc66]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:22.816958 16373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf5911377e9c4596beb72781c5f09317. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:22.820943 16241 tablet_service.cc:1467] Processing CreateTablet for tablet 322978c9a67a4ec4853b6bdaeda4a5dd (DEFAULT_TABLE table=txn_write_ops_test [id=701270f4515540f086c3f75b48e2fc66]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:22.821194 16241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 322978c9a67a4ec4853b6bdaeda4a5dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:22.821431 16464 tablet_bootstrap.cc:492] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Bootstrap starting.
I20250111 09:01:22.821751 16465 tablet_bootstrap.cc:492] T bf5911377e9c4596beb72781c5f09317 P 2cb0b3b32eb94bf688d9587b86222a0a: Bootstrap starting.
I20250111 09:01:22.823923 16464 tablet_bootstrap.cc:654] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:22.823923 16465 tablet_bootstrap.cc:654] T bf5911377e9c4596beb72781c5f09317 P 2cb0b3b32eb94bf688d9587b86222a0a: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:22.824712 16465 log.cc:826] T bf5911377e9c4596beb72781c5f09317 P 2cb0b3b32eb94bf688d9587b86222a0a: Log is configured to *not* fsync() on all Append() calls
I20250111 09:01:22.824713 16464 log.cc:826] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Log is configured to *not* fsync() on all Append() calls
I20250111 09:01:22.825374 16463 tablet_bootstrap.cc:492] T bf5911377e9c4596beb72781c5f09317 P 91e9fcf6b8ad407dbbf5c8339221b1b8: Bootstrap starting.
I20250111 09:01:22.826104 16464 tablet_bootstrap.cc:492] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: No bootstrap required, opened a new log
I20250111 09:01:22.826242 16464 ts_tablet_manager.cc:1397] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20250111 09:01:22.826246 16465 tablet_bootstrap.cc:492] T bf5911377e9c4596beb72781c5f09317 P 2cb0b3b32eb94bf688d9587b86222a0a: No bootstrap required, opened a new log
I20250111 09:01:22.826436 16465 ts_tablet_manager.cc:1397] T bf5911377e9c4596beb72781c5f09317 P 2cb0b3b32eb94bf688d9587b86222a0a: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20250111 09:01:22.828197 16463 tablet_bootstrap.cc:654] T bf5911377e9c4596beb72781c5f09317 P 91e9fcf6b8ad407dbbf5c8339221b1b8: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:22.828539 16464 raft_consensus.cc:357] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:22.828706 16464 raft_consensus.cc:383] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:22.828774 16464 raft_consensus.cc:738] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11c4b3d95f0e4ff2b317b15ff32ff0ad, State: Initialized, Role: FOLLOWER
I20250111 09:01:22.828924 16463 log.cc:826] T bf5911377e9c4596beb72781c5f09317 P 91e9fcf6b8ad407dbbf5c8339221b1b8: Log is configured to *not* fsync() on all Append() calls
I20250111 09:01:22.829171 16464 consensus_queue.cc:260] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:22.829484 16465 raft_consensus.cc:357] T bf5911377e9c4596beb72781c5f09317 P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:22.829665 16465 raft_consensus.cc:383] T bf5911377e9c4596beb72781c5f09317 P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:22.829718 16465 raft_consensus.cc:738] T bf5911377e9c4596beb72781c5f09317 P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cb0b3b32eb94bf688d9587b86222a0a, State: Initialized, Role: FOLLOWER
I20250111 09:01:22.829766 16313 heartbeater.cc:502] Master 127.14.61.62:46109 was elected leader, sending a full tablet report...
I20250111 09:01:22.829879 16464 ts_tablet_manager.cc:1428] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20250111 09:01:22.830017 16465 consensus_queue.cc:260] T bf5911377e9c4596beb72781c5f09317 P 2cb0b3b32eb94bf688d9587b86222a0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:22.830060 16464 tablet_bootstrap.cc:492] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Bootstrap starting.
I20250111 09:01:22.830526 16450 heartbeater.cc:502] Master 127.14.61.62:46109 was elected leader, sending a full tablet report...
I20250111 09:01:22.830705 16465 ts_tablet_manager.cc:1428] T bf5911377e9c4596beb72781c5f09317 P 2cb0b3b32eb94bf688d9587b86222a0a: Time spent starting tablet: real 0.004s user 0.003s sys 0.002s
I20250111 09:01:22.831020 16464 tablet_bootstrap.cc:654] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:22.831454 16465 tablet_bootstrap.cc:492] T 322978c9a67a4ec4853b6bdaeda4a5dd P 2cb0b3b32eb94bf688d9587b86222a0a: Bootstrap starting.
I20250111 09:01:22.832393 16465 tablet_bootstrap.cc:654] T 322978c9a67a4ec4853b6bdaeda4a5dd P 2cb0b3b32eb94bf688d9587b86222a0a: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:22.833498 16465 tablet_bootstrap.cc:492] T 322978c9a67a4ec4853b6bdaeda4a5dd P 2cb0b3b32eb94bf688d9587b86222a0a: No bootstrap required, opened a new log
I20250111 09:01:22.833582 16465 ts_tablet_manager.cc:1397] T 322978c9a67a4ec4853b6bdaeda4a5dd P 2cb0b3b32eb94bf688d9587b86222a0a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20250111 09:01:22.834081 16465 raft_consensus.cc:357] T 322978c9a67a4ec4853b6bdaeda4a5dd P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:22.834188 16465 raft_consensus.cc:383] T 322978c9a67a4ec4853b6bdaeda4a5dd P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:22.834203 16463 tablet_bootstrap.cc:492] T bf5911377e9c4596beb72781c5f09317 P 91e9fcf6b8ad407dbbf5c8339221b1b8: No bootstrap required, opened a new log
I20250111 09:01:22.834232 16465 raft_consensus.cc:738] T 322978c9a67a4ec4853b6bdaeda4a5dd P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cb0b3b32eb94bf688d9587b86222a0a, State: Initialized, Role: FOLLOWER
I20250111 09:01:22.834393 16463 ts_tablet_manager.cc:1397] T bf5911377e9c4596beb72781c5f09317 P 91e9fcf6b8ad407dbbf5c8339221b1b8: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20250111 09:01:22.834442 16464 tablet_bootstrap.cc:492] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad: No bootstrap required, opened a new log
I20250111 09:01:22.834507 16464 ts_tablet_manager.cc:1397] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20250111 09:01:22.834580 16465 consensus_queue.cc:260] T 322978c9a67a4ec4853b6bdaeda4a5dd P 2cb0b3b32eb94bf688d9587b86222a0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:22.834813 16465 ts_tablet_manager.cc:1428] T 322978c9a67a4ec4853b6bdaeda4a5dd P 2cb0b3b32eb94bf688d9587b86222a0a: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20250111 09:01:22.834908 16464 raft_consensus.cc:357] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:22.834978 16464 raft_consensus.cc:383] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:22.834998 16464 raft_consensus.cc:738] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11c4b3d95f0e4ff2b317b15ff32ff0ad, State: Initialized, Role: FOLLOWER
I20250111 09:01:22.835093 16464 consensus_queue.cc:260] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:22.835256 16464 ts_tablet_manager.cc:1428] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250111 09:01:22.837025 16463 raft_consensus.cc:357] T bf5911377e9c4596beb72781c5f09317 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:22.837193 16463 raft_consensus.cc:383] T bf5911377e9c4596beb72781c5f09317 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:22.837234 16463 raft_consensus.cc:738] T bf5911377e9c4596beb72781c5f09317 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e9fcf6b8ad407dbbf5c8339221b1b8, State: Initialized, Role: FOLLOWER
I20250111 09:01:22.837576 16463 consensus_queue.cc:260] T bf5911377e9c4596beb72781c5f09317 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:22.838083 16176 heartbeater.cc:502] Master 127.14.61.62:46109 was elected leader, sending a full tablet report...
I20250111 09:01:22.838371 16463 ts_tablet_manager.cc:1428] T bf5911377e9c4596beb72781c5f09317 P 91e9fcf6b8ad407dbbf5c8339221b1b8: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20250111 09:01:22.838636 16463 tablet_bootstrap.cc:492] T 322978c9a67a4ec4853b6bdaeda4a5dd P 91e9fcf6b8ad407dbbf5c8339221b1b8: Bootstrap starting.
I20250111 09:01:22.839531 16463 tablet_bootstrap.cc:654] T 322978c9a67a4ec4853b6bdaeda4a5dd P 91e9fcf6b8ad407dbbf5c8339221b1b8: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:22.840469 16463 tablet_bootstrap.cc:492] T 322978c9a67a4ec4853b6bdaeda4a5dd P 91e9fcf6b8ad407dbbf5c8339221b1b8: No bootstrap required, opened a new log
I20250111 09:01:22.840544 16463 ts_tablet_manager.cc:1397] T 322978c9a67a4ec4853b6bdaeda4a5dd P 91e9fcf6b8ad407dbbf5c8339221b1b8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:01:22.840961 16463 raft_consensus.cc:357] T 322978c9a67a4ec4853b6bdaeda4a5dd P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:22.841101 16463 raft_consensus.cc:383] T 322978c9a67a4ec4853b6bdaeda4a5dd P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:22.841141 16463 raft_consensus.cc:738] T 322978c9a67a4ec4853b6bdaeda4a5dd P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e9fcf6b8ad407dbbf5c8339221b1b8, State: Initialized, Role: FOLLOWER
I20250111 09:01:22.841264 16463 consensus_queue.cc:260] T 322978c9a67a4ec4853b6bdaeda4a5dd P 91e9fcf6b8ad407dbbf5c8339221b1b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:22.841521 16463 ts_tablet_manager.cc:1428] T 322978c9a67a4ec4853b6bdaeda4a5dd P 91e9fcf6b8ad407dbbf5c8339221b1b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20250111 09:01:22.879004 16314 tablet.cc:2367] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250111 09:01:22.905876 16469 raft_consensus.cc:491] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:01:22.906013 16469 raft_consensus.cc:513] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:22.906931 16469 leader_election.cc:290] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91e9fcf6b8ad407dbbf5c8339221b1b8 (127.14.61.1:35063), 2cb0b3b32eb94bf688d9587b86222a0a (127.14.61.3:42055)
I20250111 09:01:22.910295 16123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf5911377e9c4596beb72781c5f09317" candidate_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" is_pre_election: true
I20250111 09:01:22.910527 16123 raft_consensus.cc:2463] T bf5911377e9c4596beb72781c5f09317 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11c4b3d95f0e4ff2b317b15ff32ff0ad in term 0.
I20250111 09:01:22.910965 16193 leader_election.cc:304] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11c4b3d95f0e4ff2b317b15ff32ff0ad, 91e9fcf6b8ad407dbbf5c8339221b1b8; no voters:
I20250111 09:01:22.911165 16469 raft_consensus.cc:2798] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:01:22.911154 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf5911377e9c4596beb72781c5f09317" candidate_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" is_pre_election: true
I20250111 09:01:22.911231 16469 raft_consensus.cc:491] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:01:22.911262 16469 raft_consensus.cc:3054] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:22.911377 16399 raft_consensus.cc:2463] T bf5911377e9c4596beb72781c5f09317 P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11c4b3d95f0e4ff2b317b15ff32ff0ad in term 0.
I20250111 09:01:22.912170 16469 raft_consensus.cc:513] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:22.912461 16469 leader_election.cc:290] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [CANDIDATE]: Term 1 election: Requested vote from peers 91e9fcf6b8ad407dbbf5c8339221b1b8 (127.14.61.1:35063), 2cb0b3b32eb94bf688d9587b86222a0a (127.14.61.3:42055)
I20250111 09:01:22.912775 16123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf5911377e9c4596beb72781c5f09317" candidate_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8"
I20250111 09:01:22.912863 16123 raft_consensus.cc:3054] T bf5911377e9c4596beb72781c5f09317 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:22.912825 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf5911377e9c4596beb72781c5f09317" candidate_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb0b3b32eb94bf688d9587b86222a0a"
I20250111 09:01:22.912955 16399 raft_consensus.cc:3054] T bf5911377e9c4596beb72781c5f09317 P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:22.913662 16123 raft_consensus.cc:2463] T bf5911377e9c4596beb72781c5f09317 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11c4b3d95f0e4ff2b317b15ff32ff0ad in term 1.
I20250111 09:01:22.913769 16399 raft_consensus.cc:2463] T bf5911377e9c4596beb72781c5f09317 P 2cb0b3b32eb94bf688d9587b86222a0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11c4b3d95f0e4ff2b317b15ff32ff0ad in term 1.
I20250111 09:01:22.913913 16193 leader_election.cc:304] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11c4b3d95f0e4ff2b317b15ff32ff0ad, 91e9fcf6b8ad407dbbf5c8339221b1b8; no voters:
I20250111 09:01:22.914065 16469 raft_consensus.cc:2798] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:22.914253 16469 raft_consensus.cc:695] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 1 LEADER]: Becoming Leader. State: Replica: 11c4b3d95f0e4ff2b317b15ff32ff0ad, State: Running, Role: LEADER
I20250111 09:01:22.914495 16469 consensus_queue.cc:237] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:22.917109 15971 catalog_manager.cc:5526] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 11c4b3d95f0e4ff2b317b15ff32ff0ad (127.14.61.2). New cstate: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:22.959828 16470 raft_consensus.cc:491] T 322978c9a67a4ec4853b6bdaeda4a5dd P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:01:22.959905 16470 raft_consensus.cc:513] T 322978c9a67a4ec4853b6bdaeda4a5dd P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:22.960379 16469 raft_consensus.cc:491] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:01:22.960461 16469 raft_consensus.cc:513] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:22.960712 16470 leader_election.cc:290] T 322978c9a67a4ec4853b6bdaeda4a5dd P 2cb0b3b32eb94bf688d9587b86222a0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91e9fcf6b8ad407dbbf5c8339221b1b8 (127.14.61.1:35063), 11c4b3d95f0e4ff2b317b15ff32ff0ad (127.14.61.2:37009)
I20250111 09:01:22.961067 16469 leader_election.cc:290] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91e9fcf6b8ad407dbbf5c8339221b1b8 (127.14.61.1:35063), 2cb0b3b32eb94bf688d9587b86222a0a (127.14.61.3:42055)
I20250111 09:01:22.961055 16123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "322978c9a67a4ec4853b6bdaeda4a5dd" candidate_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" is_pre_election: true
I20250111 09:01:22.961163 16123 raft_consensus.cc:2463] T 322978c9a67a4ec4853b6bdaeda4a5dd P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11c4b3d95f0e4ff2b317b15ff32ff0ad in term 0.
I20250111 09:01:22.961410 16193 leader_election.cc:304] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11c4b3d95f0e4ff2b317b15ff32ff0ad, 91e9fcf6b8ad407dbbf5c8339221b1b8; no voters:
I20250111 09:01:22.961632 16469 raft_consensus.cc:2798] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:01:22.961649 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "322978c9a67a4ec4853b6bdaeda4a5dd" candidate_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" is_pre_election: true
I20250111 09:01:22.961730 16399 raft_consensus.cc:2463] T 322978c9a67a4ec4853b6bdaeda4a5dd P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11c4b3d95f0e4ff2b317b15ff32ff0ad in term 0.
I20250111 09:01:22.961755 16469 raft_consensus.cc:491] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:01:22.961900 16469 raft_consensus.cc:3054] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:22.962800 16469 raft_consensus.cc:513] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:22.963186 16469 leader_election.cc:290] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad [CANDIDATE]: Term 1 election: Requested vote from peers 91e9fcf6b8ad407dbbf5c8339221b1b8 (127.14.61.1:35063), 2cb0b3b32eb94bf688d9587b86222a0a (127.14.61.3:42055)
I20250111 09:01:22.963505 16123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "322978c9a67a4ec4853b6bdaeda4a5dd" candidate_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8"
I20250111 09:01:22.963506 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "322978c9a67a4ec4853b6bdaeda4a5dd" candidate_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb0b3b32eb94bf688d9587b86222a0a"
I20250111 09:01:22.963609 16123 raft_consensus.cc:3054] T 322978c9a67a4ec4853b6bdaeda4a5dd P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:22.963618 16399 raft_consensus.cc:3054] T 322978c9a67a4ec4853b6bdaeda4a5dd P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:22.964334 16125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "322978c9a67a4ec4853b6bdaeda4a5dd" candidate_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" is_pre_election: true
I20250111 09:01:22.964450 16399 raft_consensus.cc:2463] T 322978c9a67a4ec4853b6bdaeda4a5dd P 2cb0b3b32eb94bf688d9587b86222a0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11c4b3d95f0e4ff2b317b15ff32ff0ad in term 1.
I20250111 09:01:22.964514 16123 raft_consensus.cc:2463] T 322978c9a67a4ec4853b6bdaeda4a5dd P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11c4b3d95f0e4ff2b317b15ff32ff0ad in term 1.
I20250111 09:01:22.964835 16193 leader_election.cc:304] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11c4b3d95f0e4ff2b317b15ff32ff0ad, 91e9fcf6b8ad407dbbf5c8339221b1b8; no voters:
I20250111 09:01:22.965098 16469 raft_consensus.cc:2798] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:22.965198 16469 raft_consensus.cc:695] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 1 LEADER]: Becoming Leader. State: Replica: 11c4b3d95f0e4ff2b317b15ff32ff0ad, State: Running, Role: LEADER
I20250111 09:01:22.965457 16469 consensus_queue.cc:237] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:22.966110 16261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "322978c9a67a4ec4853b6bdaeda4a5dd" candidate_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" is_pre_election: true
I20250111 09:01:22.966574 16330 leader_election.cc:304] T 322978c9a67a4ec4853b6bdaeda4a5dd P 2cb0b3b32eb94bf688d9587b86222a0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2cb0b3b32eb94bf688d9587b86222a0a; no voters: 11c4b3d95f0e4ff2b317b15ff32ff0ad, 91e9fcf6b8ad407dbbf5c8339221b1b8
I20250111 09:01:22.966799 16470 raft_consensus.cc:2743] T 322978c9a67a4ec4853b6bdaeda4a5dd P 2cb0b3b32eb94bf688d9587b86222a0a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250111 09:01:22.967067 15971 catalog_manager.cc:5526] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 11c4b3d95f0e4ff2b317b15ff32ff0ad (127.14.61.2). New cstate: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:22.982712 16177 tablet.cc:2367] T 322978c9a67a4ec4853b6bdaeda4a5dd P 91e9fcf6b8ad407dbbf5c8339221b1b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250111 09:01:23.031841 15971 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33096:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001\003\000\000\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20250111 09:01:23.032294 16451 tablet.cc:2367] T 322978c9a67a4ec4853b6bdaeda4a5dd P 2cb0b3b32eb94bf688d9587b86222a0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250111 09:01:23.032352 15971 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250111 09:01:23.036208 16104 tablet_service.cc:1467] Processing CreateTablet for tablet 0debf56c8ebc4363a4aa227780e92880 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=285d52b2a84240f49a1503d8ff3df3f2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 3
I20250111 09:01:23.036326 16242 tablet_service.cc:1467] Processing CreateTablet for tablet 0debf56c8ebc4363a4aa227780e92880 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=285d52b2a84240f49a1503d8ff3df3f2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 3
I20250111 09:01:23.036485 16375 tablet_service.cc:1467] Processing CreateTablet for tablet 0debf56c8ebc4363a4aa227780e92880 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=285d52b2a84240f49a1503d8ff3df3f2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 3
I20250111 09:01:23.036737 16104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0debf56c8ebc4363a4aa227780e92880. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:23.036777 16242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0debf56c8ebc4363a4aa227780e92880. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:23.036868 16375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0debf56c8ebc4363a4aa227780e92880. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:23.038642 16464 tablet_bootstrap.cc:492] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Bootstrap starting.
I20250111 09:01:23.038986 16463 tablet_bootstrap.cc:492] T 0debf56c8ebc4363a4aa227780e92880 P 91e9fcf6b8ad407dbbf5c8339221b1b8: Bootstrap starting.
I20250111 09:01:23.039390 16465 tablet_bootstrap.cc:492] T 0debf56c8ebc4363a4aa227780e92880 P 2cb0b3b32eb94bf688d9587b86222a0a: Bootstrap starting.
I20250111 09:01:23.039557 16464 tablet_bootstrap.cc:654] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:23.039880 16463 tablet_bootstrap.cc:654] T 0debf56c8ebc4363a4aa227780e92880 P 91e9fcf6b8ad407dbbf5c8339221b1b8: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:23.040076 16465 tablet_bootstrap.cc:654] T 0debf56c8ebc4363a4aa227780e92880 P 2cb0b3b32eb94bf688d9587b86222a0a: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:23.040580 16464 tablet_bootstrap.cc:492] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: No bootstrap required, opened a new log
I20250111 09:01:23.040657 16464 ts_tablet_manager.cc:1397] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20250111 09:01:23.041044 16465 tablet_bootstrap.cc:492] T 0debf56c8ebc4363a4aa227780e92880 P 2cb0b3b32eb94bf688d9587b86222a0a: No bootstrap required, opened a new log
I20250111 09:01:23.041121 16465 ts_tablet_manager.cc:1397] T 0debf56c8ebc4363a4aa227780e92880 P 2cb0b3b32eb94bf688d9587b86222a0a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20250111 09:01:23.041098 16464 raft_consensus.cc:357] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.041179 16464 raft_consensus.cc:383] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:23.041211 16464 raft_consensus.cc:738] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11c4b3d95f0e4ff2b317b15ff32ff0ad, State: Initialized, Role: FOLLOWER
I20250111 09:01:23.041319 16464 consensus_queue.cc:260] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.041383 16463 tablet_bootstrap.cc:492] T 0debf56c8ebc4363a4aa227780e92880 P 91e9fcf6b8ad407dbbf5c8339221b1b8: No bootstrap required, opened a new log
I20250111 09:01:23.041448 16463 ts_tablet_manager.cc:1397] T 0debf56c8ebc4363a4aa227780e92880 P 91e9fcf6b8ad407dbbf5c8339221b1b8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250111 09:01:23.041548 16464 ts_tablet_manager.cc:1428] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:01:23.041524 16465 raft_consensus.cc:357] T 0debf56c8ebc4363a4aa227780e92880 P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.041625 16465 raft_consensus.cc:383] T 0debf56c8ebc4363a4aa227780e92880 P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:23.041656 16465 raft_consensus.cc:738] T 0debf56c8ebc4363a4aa227780e92880 P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cb0b3b32eb94bf688d9587b86222a0a, State: Initialized, Role: FOLLOWER
I20250111 09:01:23.041759 16465 consensus_queue.cc:260] T 0debf56c8ebc4363a4aa227780e92880 P 2cb0b3b32eb94bf688d9587b86222a0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.041777 16469 tablet_replica.cc:440] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.041867 16469 tablet_replica.cc:443] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.041869 16463 raft_consensus.cc:357] T 0debf56c8ebc4363a4aa227780e92880 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.041946 16463 raft_consensus.cc:383] T 0debf56c8ebc4363a4aa227780e92880 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:23.041965 16463 raft_consensus.cc:738] T 0debf56c8ebc4363a4aa227780e92880 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e9fcf6b8ad407dbbf5c8339221b1b8, State: Initialized, Role: FOLLOWER
I20250111 09:01:23.042019 16465 ts_tablet_manager.cc:1428] T 0debf56c8ebc4363a4aa227780e92880 P 2cb0b3b32eb94bf688d9587b86222a0a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:01:23.042052 16463 consensus_queue.cc:260] T 0debf56c8ebc4363a4aa227780e92880 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.042244 16470 tablet_replica.cc:440] T 0debf56c8ebc4363a4aa227780e92880 P 2cb0b3b32eb94bf688d9587b86222a0a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.042284 16463 ts_tablet_manager.cc:1428] T 0debf56c8ebc4363a4aa227780e92880 P 91e9fcf6b8ad407dbbf5c8339221b1b8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250111 09:01:23.042333 16470 tablet_replica.cc:443] T 0debf56c8ebc4363a4aa227780e92880 P 2cb0b3b32eb94bf688d9587b86222a0a: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.042508 16473 tablet_replica.cc:440] T 0debf56c8ebc4363a4aa227780e92880 P 91e9fcf6b8ad407dbbf5c8339221b1b8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.042604 16473 tablet_replica.cc:443] T 0debf56c8ebc4363a4aa227780e92880 P 91e9fcf6b8ad407dbbf5c8339221b1b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.052889 16469 raft_consensus.cc:491] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:01:23.052978 16469 raft_consensus.cc:513] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.053268 16469 leader_election.cc:290] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91e9fcf6b8ad407dbbf5c8339221b1b8 (127.14.61.1:35063), 2cb0b3b32eb94bf688d9587b86222a0a (127.14.61.3:42055)
I20250111 09:01:23.053544 16125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0debf56c8ebc4363a4aa227780e92880" candidate_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" is_pre_election: true
I20250111 09:01:23.053630 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0debf56c8ebc4363a4aa227780e92880" candidate_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" is_pre_election: true
I20250111 09:01:23.053666 16125 raft_consensus.cc:2463] T 0debf56c8ebc4363a4aa227780e92880 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11c4b3d95f0e4ff2b317b15ff32ff0ad in term 0.
I20250111 09:01:23.053738 16399 raft_consensus.cc:2463] T 0debf56c8ebc4363a4aa227780e92880 P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11c4b3d95f0e4ff2b317b15ff32ff0ad in term 0.
I20250111 09:01:23.053901 16193 leader_election.cc:304] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11c4b3d95f0e4ff2b317b15ff32ff0ad, 91e9fcf6b8ad407dbbf5c8339221b1b8; no voters:
I20250111 09:01:23.054082 16469 raft_consensus.cc:2798] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:01:23.054129 16469 raft_consensus.cc:491] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:01:23.054174 16469 raft_consensus.cc:3054] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:23.054795 16469 raft_consensus.cc:513] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.055052 16469 leader_election.cc:290] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [CANDIDATE]: Term 1 election: Requested vote from peers 91e9fcf6b8ad407dbbf5c8339221b1b8 (127.14.61.1:35063), 2cb0b3b32eb94bf688d9587b86222a0a (127.14.61.3:42055)
I20250111 09:01:23.055310 16125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0debf56c8ebc4363a4aa227780e92880" candidate_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8"
I20250111 09:01:23.055359 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0debf56c8ebc4363a4aa227780e92880" candidate_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb0b3b32eb94bf688d9587b86222a0a"
I20250111 09:01:23.055436 16125 raft_consensus.cc:3054] T 0debf56c8ebc4363a4aa227780e92880 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:23.055459 16399 raft_consensus.cc:3054] T 0debf56c8ebc4363a4aa227780e92880 P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:23.056201 16399 raft_consensus.cc:2463] T 0debf56c8ebc4363a4aa227780e92880 P 2cb0b3b32eb94bf688d9587b86222a0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11c4b3d95f0e4ff2b317b15ff32ff0ad in term 1.
I20250111 09:01:23.056205 16125 raft_consensus.cc:2463] T 0debf56c8ebc4363a4aa227780e92880 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11c4b3d95f0e4ff2b317b15ff32ff0ad in term 1.
I20250111 09:01:23.056422 16193 leader_election.cc:304] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11c4b3d95f0e4ff2b317b15ff32ff0ad, 91e9fcf6b8ad407dbbf5c8339221b1b8; no voters:
I20250111 09:01:23.056558 16469 raft_consensus.cc:2798] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:23.056638 16469 raft_consensus.cc:695] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 1 LEADER]: Becoming Leader. State: Replica: 11c4b3d95f0e4ff2b317b15ff32ff0ad, State: Running, Role: LEADER
I20250111 09:01:23.056758 16469 consensus_queue.cc:237] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.057317 16476 tablet_replica.cc:440] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: TxnStatusTablet state changed. Reason: New leader 11c4b3d95f0e4ff2b317b15ff32ff0ad. Latest consensus state: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.057403 16476 tablet_replica.cc:443] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:23.057644 16491 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:01:23.057811 16491 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:01:23.058045 15971 catalog_manager.cc:5526] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 11c4b3d95f0e4ff2b317b15ff32ff0ad (127.14.61.2). New cstate: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:23.072384 16125 raft_consensus.cc:1270] T 0debf56c8ebc4363a4aa227780e92880 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 1 FOLLOWER]: Refusing update from remote peer 11c4b3d95f0e4ff2b317b15ff32ff0ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:23.072391 16399 raft_consensus.cc:1270] T 0debf56c8ebc4363a4aa227780e92880 P 2cb0b3b32eb94bf688d9587b86222a0a [term 1 FOLLOWER]: Refusing update from remote peer 11c4b3d95f0e4ff2b317b15ff32ff0ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:23.072902 16469 consensus_queue.cc:1035] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:23.073050 16476 consensus_queue.cc:1035] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:23.075981 16470 tablet_replica.cc:440] T 0debf56c8ebc4363a4aa227780e92880 P 2cb0b3b32eb94bf688d9587b86222a0a: TxnStatusTablet state changed. Reason: New leader 11c4b3d95f0e4ff2b317b15ff32ff0ad. Latest consensus state: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.076076 16470 tablet_replica.cc:443] T 0debf56c8ebc4363a4aa227780e92880 P 2cb0b3b32eb94bf688d9587b86222a0a: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.076404 16473 tablet_replica.cc:440] T 0debf56c8ebc4363a4aa227780e92880 P 91e9fcf6b8ad407dbbf5c8339221b1b8: TxnStatusTablet state changed. Reason: New leader 11c4b3d95f0e4ff2b317b15ff32ff0ad. Latest consensus state: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.076485 16473 tablet_replica.cc:443] T 0debf56c8ebc4363a4aa227780e92880 P 91e9fcf6b8ad407dbbf5c8339221b1b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.078572 16476 tablet_replica.cc:440] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.078663 16476 tablet_replica.cc:443] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:23.078994 16476 tablet_replica.cc:440] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.079082 16476 tablet_replica.cc:443] T 0debf56c8ebc4363a4aa227780e92880 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:23.079493 16473 tablet_replica.cc:440] T 0debf56c8ebc4363a4aa227780e92880 P 91e9fcf6b8ad407dbbf5c8339221b1b8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.079584 16473 tablet_replica.cc:443] T 0debf56c8ebc4363a4aa227780e92880 P 91e9fcf6b8ad407dbbf5c8339221b1b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.081219 16470 tablet_replica.cc:440] T 0debf56c8ebc4363a4aa227780e92880 P 2cb0b3b32eb94bf688d9587b86222a0a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.081291 16470 tablet_replica.cc:443] T 0debf56c8ebc4363a4aa227780e92880 P 2cb0b3b32eb94bf688d9587b86222a0a: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.082034 16496 mvcc.cc:204] Tried to move back new op lower bound from 7113056596259942400 to 7113056596201422848. Current Snapshot: MvccSnapshot[applied={T|T < 7113056596259942400 or (T in {7113056596259942400})}]
I20250111 09:01:23.089910 16495 mvcc.cc:204] Tried to move back new op lower bound from 7113056596259942400 to 7113056596201422848. Current Snapshot: MvccSnapshot[applied={T|T < 7113056596259942400}]
I20250111 09:01:23.101847 16399 raft_consensus.cc:1270] T bf5911377e9c4596beb72781c5f09317 P 2cb0b3b32eb94bf688d9587b86222a0a [term 1 FOLLOWER]: Refusing update from remote peer 11c4b3d95f0e4ff2b317b15ff32ff0ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:23.101913 16125 raft_consensus.cc:1270] T bf5911377e9c4596beb72781c5f09317 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 1 FOLLOWER]: Refusing update from remote peer 11c4b3d95f0e4ff2b317b15ff32ff0ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:23.102208 16505 consensus_queue.cc:1035] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:23.102360 16476 consensus_queue.cc:1035] T bf5911377e9c4596beb72781c5f09317 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:23.104601 16494 mvcc.cc:204] Tried to move back new op lower bound from 7113056596382699520 to 7113056595620012032. Current Snapshot: MvccSnapshot[applied={T|T < 7113056596382699520}]
I20250111 09:01:23.140147 16125 raft_consensus.cc:1270] T 322978c9a67a4ec4853b6bdaeda4a5dd P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 1 FOLLOWER]: Refusing update from remote peer 11c4b3d95f0e4ff2b317b15ff32ff0ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:23.140166 16399 raft_consensus.cc:1270] T 322978c9a67a4ec4853b6bdaeda4a5dd P 2cb0b3b32eb94bf688d9587b86222a0a [term 1 FOLLOWER]: Refusing update from remote peer 11c4b3d95f0e4ff2b317b15ff32ff0ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:23.140460 16469 consensus_queue.cc:1035] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:23.140578 16509 consensus_queue.cc:1035] T 322978c9a67a4ec4853b6bdaeda4a5dd P 11c4b3d95f0e4ff2b317b15ff32ff0ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:23.195333 15971 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33096:
table { table_name: "kudu_system.kudu_transactions" } alter_schema_steps { type: ADD_RANGE_PARTITION add_range_partition { range_bounds { rows: "<redacted>""\006\001\003\000\000\000\000\000\000\000\007\001\006\000\000\000\000\000\000\000" indirect_data: "<redacted>""" } } } schema { columns { name: "txn_id" type: INT64 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "entry_type" type: INT8 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "identifier" type: STRING is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "metadata" type: STRING is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } modify_external_catalogs: false
W20250111 09:01:23.198100 15971 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dfc9416bf7a14fe3bdd1db2477aa6f98 has no consensus state
I20250111 09:01:23.200600 16104 tablet_service.cc:1467] Processing CreateTablet for tablet dfc9416bf7a14fe3bdd1db2477aa6f98 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=285d52b2a84240f49a1503d8ff3df3f2]), partition=RANGE (txn_id) PARTITION 3 <= VALUES < 6
I20250111 09:01:23.200860 16104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfc9416bf7a14fe3bdd1db2477aa6f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:23.200855 16375 tablet_service.cc:1467] Processing CreateTablet for tablet dfc9416bf7a14fe3bdd1db2477aa6f98 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=285d52b2a84240f49a1503d8ff3df3f2]), partition=RANGE (txn_id) PARTITION 3 <= VALUES < 6
I20250111 09:01:23.201093 16375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfc9416bf7a14fe3bdd1db2477aa6f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:23.202873 16463 tablet_bootstrap.cc:492] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8: Bootstrap starting.
I20250111 09:01:23.202891 16465 tablet_bootstrap.cc:492] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 2cb0b3b32eb94bf688d9587b86222a0a: Bootstrap starting.
I20250111 09:01:23.203886 16463 tablet_bootstrap.cc:654] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:23.204048 16465 tablet_bootstrap.cc:654] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 2cb0b3b32eb94bf688d9587b86222a0a: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:23.204862 16242 tablet_service.cc:1467] Processing CreateTablet for tablet dfc9416bf7a14fe3bdd1db2477aa6f98 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=285d52b2a84240f49a1503d8ff3df3f2]), partition=RANGE (txn_id) PARTITION 3 <= VALUES < 6
I20250111 09:01:23.204989 16463 tablet_bootstrap.cc:492] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8: No bootstrap required, opened a new log
I20250111 09:01:23.205109 16463 ts_tablet_manager.cc:1397] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:01:23.205113 16242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfc9416bf7a14fe3bdd1db2477aa6f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:23.205539 16463 raft_consensus.cc:357] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.205641 16463 raft_consensus.cc:383] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:23.205674 16463 raft_consensus.cc:738] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e9fcf6b8ad407dbbf5c8339221b1b8, State: Initialized, Role: FOLLOWER
I20250111 09:01:23.205816 16463 consensus_queue.cc:260] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.205992 16473 tablet_replica.cc:440] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.206040 16463 ts_tablet_manager.cc:1428] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250111 09:01:23.206066 16473 tablet_replica.cc:443] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.207463 16465 tablet_bootstrap.cc:492] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 2cb0b3b32eb94bf688d9587b86222a0a: No bootstrap required, opened a new log
I20250111 09:01:23.207538 16465 ts_tablet_manager.cc:1397] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 2cb0b3b32eb94bf688d9587b86222a0a: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20250111 09:01:23.207670 16464 tablet_bootstrap.cc:492] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Bootstrap starting.
I20250111 09:01:23.207921 16465 raft_consensus.cc:357] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.208015 16465 raft_consensus.cc:383] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:23.208046 16465 raft_consensus.cc:738] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cb0b3b32eb94bf688d9587b86222a0a, State: Initialized, Role: FOLLOWER
I20250111 09:01:23.208158 16465 consensus_queue.cc:260] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 2cb0b3b32eb94bf688d9587b86222a0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.208407 16465 ts_tablet_manager.cc:1428] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 2cb0b3b32eb94bf688d9587b86222a0a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:01:23.208541 16464 tablet_bootstrap.cc:654] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:23.208530 16470 tablet_replica.cc:440] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 2cb0b3b32eb94bf688d9587b86222a0a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.208595 16470 tablet_replica.cc:443] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 2cb0b3b32eb94bf688d9587b86222a0a: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.210155 16464 tablet_bootstrap.cc:492] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: No bootstrap required, opened a new log
I20250111 09:01:23.210230 16464 ts_tablet_manager.cc:1397] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20250111 09:01:23.210301 15971 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6: kudu_system.kudu_transactions [id=285d52b2a84240f49a1503d8ff3df3f2] alter complete (version 0)
I20250111 09:01:23.210672 16464 raft_consensus.cc:357] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.210762 16464 raft_consensus.cc:383] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:23.210801 16464 raft_consensus.cc:738] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11c4b3d95f0e4ff2b317b15ff32ff0ad, State: Initialized, Role: FOLLOWER
I20250111 09:01:23.210899 16464 consensus_queue.cc:260] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.211153 16464 ts_tablet_manager.cc:1428] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20250111 09:01:23.211453 16469 tablet_replica.cc:440] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.211593 16469 tablet_replica.cc:443] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.233065 16473 raft_consensus.cc:491] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:01:23.233167 16473 raft_consensus.cc:513] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.234074 16473 leader_election.cc:290] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11c4b3d95f0e4ff2b317b15ff32ff0ad (127.14.61.2:37009), 2cb0b3b32eb94bf688d9587b86222a0a (127.14.61.3:42055)
I20250111 09:01:23.238303 16261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfc9416bf7a14fe3bdd1db2477aa6f98" candidate_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" is_pre_election: true
I20250111 09:01:23.238469 16261 raft_consensus.cc:2463] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e9fcf6b8ad407dbbf5c8339221b1b8 in term 0.
I20250111 09:01:23.238502 16398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfc9416bf7a14fe3bdd1db2477aa6f98" candidate_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" is_pre_election: true
I20250111 09:01:23.238619 16398 raft_consensus.cc:2463] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e9fcf6b8ad407dbbf5c8339221b1b8 in term 0.
I20250111 09:01:23.238903 16056 leader_election.cc:304] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cb0b3b32eb94bf688d9587b86222a0a, 91e9fcf6b8ad407dbbf5c8339221b1b8; no voters:
I20250111 09:01:23.239080 16473 raft_consensus.cc:2798] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:01:23.239131 16473 raft_consensus.cc:491] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:01:23.239164 16473 raft_consensus.cc:3054] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:23.239861 16473 raft_consensus.cc:513] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.240099 16473 leader_election.cc:290] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [CANDIDATE]: Term 1 election: Requested vote from peers 11c4b3d95f0e4ff2b317b15ff32ff0ad (127.14.61.2:37009), 2cb0b3b32eb94bf688d9587b86222a0a (127.14.61.3:42055)
I20250111 09:01:23.240410 16398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfc9416bf7a14fe3bdd1db2477aa6f98" candidate_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb0b3b32eb94bf688d9587b86222a0a"
I20250111 09:01:23.240497 16398 raft_consensus.cc:3054] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:23.241240 16398 raft_consensus.cc:2463] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 2cb0b3b32eb94bf688d9587b86222a0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e9fcf6b8ad407dbbf5c8339221b1b8 in term 1.
I20250111 09:01:23.241356 16261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfc9416bf7a14fe3bdd1db2477aa6f98" candidate_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad"
I20250111 09:01:23.241461 16261 raft_consensus.cc:3054] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:23.242280 16261 raft_consensus.cc:2463] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e9fcf6b8ad407dbbf5c8339221b1b8 in term 1.
I20250111 09:01:23.242597 16056 leader_election.cc:304] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cb0b3b32eb94bf688d9587b86222a0a, 91e9fcf6b8ad407dbbf5c8339221b1b8; no voters:
I20250111 09:01:23.242806 16473 raft_consensus.cc:2798] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:23.242995 16473 raft_consensus.cc:695] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 1 LEADER]: Becoming Leader. State: Replica: 91e9fcf6b8ad407dbbf5c8339221b1b8, State: Running, Role: LEADER
I20250111 09:01:23.243268 16473 consensus_queue.cc:237] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.244371 16517 tablet_replica.cc:440] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8: TxnStatusTablet state changed. Reason: New leader 91e9fcf6b8ad407dbbf5c8339221b1b8. Latest consensus state: current_term: 1 leader_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.244490 16517 tablet_replica.cc:443] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:23.246577 16519 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:01:23.246780 16519 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:01:23.248054 15969 catalog_manager.cc:5526] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91e9fcf6b8ad407dbbf5c8339221b1b8 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:23.264000 16398 raft_consensus.cc:1270] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 2cb0b3b32eb94bf688d9587b86222a0a [term 1 FOLLOWER]: Refusing update from remote peer 91e9fcf6b8ad407dbbf5c8339221b1b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:23.264164 16261 raft_consensus.cc:1270] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 1 FOLLOWER]: Refusing update from remote peer 91e9fcf6b8ad407dbbf5c8339221b1b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:23.264389 16517 consensus_queue.cc:1035] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:23.264648 16473 consensus_queue.cc:1035] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:23.266139 16470 tablet_replica.cc:440] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 2cb0b3b32eb94bf688d9587b86222a0a: TxnStatusTablet state changed. Reason: New leader 91e9fcf6b8ad407dbbf5c8339221b1b8. Latest consensus state: current_term: 1 leader_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.266223 16470 tablet_replica.cc:443] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 2cb0b3b32eb94bf688d9587b86222a0a: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.266207 16469 tablet_replica.cc:440] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: TxnStatusTablet state changed. Reason: New leader 91e9fcf6b8ad407dbbf5c8339221b1b8. Latest consensus state: current_term: 1 leader_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.266283 16469 tablet_replica.cc:443] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.267727 16473 tablet_replica.cc:440] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.267812 16473 tablet_replica.cc:443] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:23.268779 16470 tablet_replica.cc:440] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 2cb0b3b32eb94bf688d9587b86222a0a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.268855 16470 tablet_replica.cc:443] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 2cb0b3b32eb94bf688d9587b86222a0a: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.269279 16473 tablet_replica.cc:440] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.269367 16473 tablet_replica.cc:443] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 91e9fcf6b8ad407dbbf5c8339221b1b8: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:23.271188 16469 tablet_replica.cc:440] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.271282 16469 tablet_replica.cc:443] T dfc9416bf7a14fe3bdd1db2477aa6f98 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.382738 15970 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33096:
table { table_name: "kudu_system.kudu_transactions" } alter_schema_steps { type: ADD_RANGE_PARTITION add_range_partition { range_bounds { rows: "<redacted>""\006\001\006\000\000\000\000\000\000\000\007\001\t\000\000\000\000\000\000\000" indirect_data: "<redacted>""" } } } schema { columns { name: "txn_id" type: INT64 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "entry_type" type: INT8 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "identifier" type: STRING is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "metadata" type: STRING is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } modify_external_catalogs: false
W20250111 09:01:23.385756 15970 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 76fe4a80ac1148bfb53805e635de7746 has no consensus state
I20250111 09:01:23.388299 16104 tablet_service.cc:1467] Processing CreateTablet for tablet 76fe4a80ac1148bfb53805e635de7746 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=285d52b2a84240f49a1503d8ff3df3f2]), partition=RANGE (txn_id) PARTITION 6 <= VALUES < 9
I20250111 09:01:23.388300 16242 tablet_service.cc:1467] Processing CreateTablet for tablet 76fe4a80ac1148bfb53805e635de7746 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=285d52b2a84240f49a1503d8ff3df3f2]), partition=RANGE (txn_id) PARTITION 6 <= VALUES < 9
I20250111 09:01:23.388566 16242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76fe4a80ac1148bfb53805e635de7746. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:23.388592 16104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76fe4a80ac1148bfb53805e635de7746. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:23.388540 16375 tablet_service.cc:1467] Processing CreateTablet for tablet 76fe4a80ac1148bfb53805e635de7746 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=285d52b2a84240f49a1503d8ff3df3f2]), partition=RANGE (txn_id) PARTITION 6 <= VALUES < 9
I20250111 09:01:23.388794 16375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76fe4a80ac1148bfb53805e635de7746. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:23.390486 16463 tablet_bootstrap.cc:492] T 76fe4a80ac1148bfb53805e635de7746 P 91e9fcf6b8ad407dbbf5c8339221b1b8: Bootstrap starting.
I20250111 09:01:23.390556 16464 tablet_bootstrap.cc:492] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Bootstrap starting.
I20250111 09:01:23.390854 16465 tablet_bootstrap.cc:492] T 76fe4a80ac1148bfb53805e635de7746 P 2cb0b3b32eb94bf688d9587b86222a0a: Bootstrap starting.
I20250111 09:01:23.391364 16463 tablet_bootstrap.cc:654] T 76fe4a80ac1148bfb53805e635de7746 P 91e9fcf6b8ad407dbbf5c8339221b1b8: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:23.391364 16464 tablet_bootstrap.cc:654] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:23.391808 16465 tablet_bootstrap.cc:654] T 76fe4a80ac1148bfb53805e635de7746 P 2cb0b3b32eb94bf688d9587b86222a0a: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:23.392613 16464 tablet_bootstrap.cc:492] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: No bootstrap required, opened a new log
I20250111 09:01:23.392688 16464 ts_tablet_manager.cc:1397] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20250111 09:01:23.392865 16463 tablet_bootstrap.cc:492] T 76fe4a80ac1148bfb53805e635de7746 P 91e9fcf6b8ad407dbbf5c8339221b1b8: No bootstrap required, opened a new log
I20250111 09:01:23.392949 16463 ts_tablet_manager.cc:1397] T 76fe4a80ac1148bfb53805e635de7746 P 91e9fcf6b8ad407dbbf5c8339221b1b8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20250111 09:01:23.393149 16464 raft_consensus.cc:357] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.393245 16464 raft_consensus.cc:383] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:23.393278 16464 raft_consensus.cc:738] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11c4b3d95f0e4ff2b317b15ff32ff0ad, State: Initialized, Role: FOLLOWER
I20250111 09:01:23.393390 16464 consensus_queue.cc:260] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.393401 16463 raft_consensus.cc:357] T 76fe4a80ac1148bfb53805e635de7746 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.393481 16463 raft_consensus.cc:383] T 76fe4a80ac1148bfb53805e635de7746 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:23.393512 16463 raft_consensus.cc:738] T 76fe4a80ac1148bfb53805e635de7746 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e9fcf6b8ad407dbbf5c8339221b1b8, State: Initialized, Role: FOLLOWER
I20250111 09:01:23.393656 16464 ts_tablet_manager.cc:1428] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250111 09:01:23.393615 16463 consensus_queue.cc:260] T 76fe4a80ac1148bfb53805e635de7746 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.393666 16465 tablet_bootstrap.cc:492] T 76fe4a80ac1148bfb53805e635de7746 P 2cb0b3b32eb94bf688d9587b86222a0a: No bootstrap required, opened a new log
I20250111 09:01:23.393743 16465 ts_tablet_manager.cc:1397] T 76fe4a80ac1148bfb53805e635de7746 P 2cb0b3b32eb94bf688d9587b86222a0a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250111 09:01:23.393887 16463 ts_tablet_manager.cc:1428] T 76fe4a80ac1148bfb53805e635de7746 P 91e9fcf6b8ad407dbbf5c8339221b1b8: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250111 09:01:23.394098 16505 tablet_replica.cc:440] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.394171 16505 tablet_replica.cc:443] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.394171 16465 raft_consensus.cc:357] T 76fe4a80ac1148bfb53805e635de7746 P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.394264 16465 raft_consensus.cc:383] T 76fe4a80ac1148bfb53805e635de7746 P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:23.394292 16465 raft_consensus.cc:738] T 76fe4a80ac1148bfb53805e635de7746 P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cb0b3b32eb94bf688d9587b86222a0a, State: Initialized, Role: FOLLOWER
I20250111 09:01:23.394315 16523 tablet_replica.cc:440] T 76fe4a80ac1148bfb53805e635de7746 P 91e9fcf6b8ad407dbbf5c8339221b1b8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.394387 16523 tablet_replica.cc:443] T 76fe4a80ac1148bfb53805e635de7746 P 91e9fcf6b8ad407dbbf5c8339221b1b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.394389 16465 consensus_queue.cc:260] T 76fe4a80ac1148bfb53805e635de7746 P 2cb0b3b32eb94bf688d9587b86222a0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.394626 16465 ts_tablet_manager.cc:1428] T 76fe4a80ac1148bfb53805e635de7746 P 2cb0b3b32eb94bf688d9587b86222a0a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:01:23.394850 16470 tablet_replica.cc:440] T 76fe4a80ac1148bfb53805e635de7746 P 2cb0b3b32eb94bf688d9587b86222a0a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.394946 16470 tablet_replica.cc:443] T 76fe4a80ac1148bfb53805e635de7746 P 2cb0b3b32eb94bf688d9587b86222a0a: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.396152 15970 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6: kudu_system.kudu_transactions [id=285d52b2a84240f49a1503d8ff3df3f2] alter complete (version 0)
W20250111 09:01:23.413992 15958 catalog_manager.cc:4670] Async tablet task AlterTable RPC for tablet 76fe4a80ac1148bfb53805e635de7746 (table kudu_system.kudu_transactions [id=285d52b2a84240f49a1503d8ff3df3f2], current schema version=0) failed: Not found: failed to reset TS proxy: No leader found: Tablet 76fe4a80ac1148bfb53805e635de7746 consensus state has no leader
I20250111 09:01:23.803786 16505 raft_consensus.cc:491] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:01:23.803889 16505 raft_consensus.cc:513] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.804226 16505 leader_election.cc:290] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91e9fcf6b8ad407dbbf5c8339221b1b8 (127.14.61.1:35063), 2cb0b3b32eb94bf688d9587b86222a0a (127.14.61.3:42055)
I20250111 09:01:23.804569 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fe4a80ac1148bfb53805e635de7746" candidate_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" is_pre_election: true
I20250111 09:01:23.804530 16123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fe4a80ac1148bfb53805e635de7746" candidate_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" is_pre_election: true
I20250111 09:01:23.804718 16399 raft_consensus.cc:2463] T 76fe4a80ac1148bfb53805e635de7746 P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11c4b3d95f0e4ff2b317b15ff32ff0ad in term 0.
I20250111 09:01:23.804701 16123 raft_consensus.cc:2463] T 76fe4a80ac1148bfb53805e635de7746 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11c4b3d95f0e4ff2b317b15ff32ff0ad in term 0.
I20250111 09:01:23.804988 16193 leader_election.cc:304] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11c4b3d95f0e4ff2b317b15ff32ff0ad, 91e9fcf6b8ad407dbbf5c8339221b1b8; no voters:
I20250111 09:01:23.805162 16505 raft_consensus.cc:2798] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:01:23.805225 16505 raft_consensus.cc:491] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:01:23.805279 16505 raft_consensus.cc:3054] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:23.806236 16505 raft_consensus.cc:513] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.806512 16505 leader_election.cc:290] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [CANDIDATE]: Term 1 election: Requested vote from peers 91e9fcf6b8ad407dbbf5c8339221b1b8 (127.14.61.1:35063), 2cb0b3b32eb94bf688d9587b86222a0a (127.14.61.3:42055)
I20250111 09:01:23.806807 16123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fe4a80ac1148bfb53805e635de7746" candidate_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8"
I20250111 09:01:23.806823 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fe4a80ac1148bfb53805e635de7746" candidate_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb0b3b32eb94bf688d9587b86222a0a"
I20250111 09:01:23.806917 16399 raft_consensus.cc:3054] T 76fe4a80ac1148bfb53805e635de7746 P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:23.806917 16123 raft_consensus.cc:3054] T 76fe4a80ac1148bfb53805e635de7746 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:23.807873 16399 raft_consensus.cc:2463] T 76fe4a80ac1148bfb53805e635de7746 P 2cb0b3b32eb94bf688d9587b86222a0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11c4b3d95f0e4ff2b317b15ff32ff0ad in term 1.
I20250111 09:01:23.807869 16123 raft_consensus.cc:2463] T 76fe4a80ac1148bfb53805e635de7746 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11c4b3d95f0e4ff2b317b15ff32ff0ad in term 1.
I20250111 09:01:23.808094 16193 leader_election.cc:304] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11c4b3d95f0e4ff2b317b15ff32ff0ad, 2cb0b3b32eb94bf688d9587b86222a0a; no voters:
I20250111 09:01:23.808247 16505 raft_consensus.cc:2798] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:23.808327 16505 raft_consensus.cc:695] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 1 LEADER]: Becoming Leader. State: Replica: 11c4b3d95f0e4ff2b317b15ff32ff0ad, State: Running, Role: LEADER
I20250111 09:01:23.808445 16505 consensus_queue.cc:237] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.808990 16509 tablet_replica.cc:440] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: TxnStatusTablet state changed. Reason: New leader 11c4b3d95f0e4ff2b317b15ff32ff0ad. Latest consensus state: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.809105 16509 tablet_replica.cc:443] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:23.809262 16533 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:01:23.809347 16533 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:01:23.809922 15971 catalog_manager.cc:5526] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 11c4b3d95f0e4ff2b317b15ff32ff0ad (127.14.61.2). New cstate: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:23.824012 16399 raft_consensus.cc:1270] T 76fe4a80ac1148bfb53805e635de7746 P 2cb0b3b32eb94bf688d9587b86222a0a [term 1 FOLLOWER]: Refusing update from remote peer 11c4b3d95f0e4ff2b317b15ff32ff0ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:23.824025 16123 raft_consensus.cc:1270] T 76fe4a80ac1148bfb53805e635de7746 P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 1 FOLLOWER]: Refusing update from remote peer 11c4b3d95f0e4ff2b317b15ff32ff0ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:23.824303 16509 consensus_queue.cc:1035] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:23.824393 16505 consensus_queue.cc:1035] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:23.825603 16470 tablet_replica.cc:440] T 76fe4a80ac1148bfb53805e635de7746 P 2cb0b3b32eb94bf688d9587b86222a0a: TxnStatusTablet state changed. Reason: New leader 11c4b3d95f0e4ff2b317b15ff32ff0ad. Latest consensus state: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.825708 16470 tablet_replica.cc:443] T 76fe4a80ac1148bfb53805e635de7746 P 2cb0b3b32eb94bf688d9587b86222a0a: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.826448 16523 tablet_replica.cc:440] T 76fe4a80ac1148bfb53805e635de7746 P 91e9fcf6b8ad407dbbf5c8339221b1b8: TxnStatusTablet state changed. Reason: New leader 11c4b3d95f0e4ff2b317b15ff32ff0ad. Latest consensus state: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.826547 16523 tablet_replica.cc:443] T 76fe4a80ac1148bfb53805e635de7746 P 91e9fcf6b8ad407dbbf5c8339221b1b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.827950 16509 tablet_replica.cc:440] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.828025 16509 tablet_replica.cc:443] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:23.828428 16470 tablet_replica.cc:440] T 76fe4a80ac1148bfb53805e635de7746 P 2cb0b3b32eb94bf688d9587b86222a0a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.828505 16470 tablet_replica.cc:443] T 76fe4a80ac1148bfb53805e635de7746 P 2cb0b3b32eb94bf688d9587b86222a0a: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.829190 16469 tablet_replica.cc:440] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.829258 16469 tablet_replica.cc:443] T 76fe4a80ac1148bfb53805e635de7746 P 11c4b3d95f0e4ff2b317b15ff32ff0ad: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:23.830166 16523 tablet_replica.cc:440] T 76fe4a80ac1148bfb53805e635de7746 P 91e9fcf6b8ad407dbbf5c8339221b1b8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.830238 16523 tablet_replica.cc:443] T 76fe4a80ac1148bfb53805e635de7746 P 91e9fcf6b8ad407dbbf5c8339221b1b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.910660 15970 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33096:
table { table_name: "kudu_system.kudu_transactions" } alter_schema_steps { type: ADD_RANGE_PARTITION add_range_partition { range_bounds { rows: "<redacted>""\006\001\t\000\000\000\000\000\000\000\007\001\014\000\000\000\000\000\000\000" indirect_data: "<redacted>""" } } } schema { columns { name: "txn_id" type: INT64 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "entry_type" type: INT8 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "identifier" type: STRING is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "metadata" type: STRING is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } modify_external_catalogs: false
W20250111 09:01:23.912783 15970 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 317a27b34d394490b909c5090de89aaa has no consensus state
I20250111 09:01:23.915185 16104 tablet_service.cc:1467] Processing CreateTablet for tablet 317a27b34d394490b909c5090de89aaa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=285d52b2a84240f49a1503d8ff3df3f2]), partition=RANGE (txn_id) PARTITION 9 <= VALUES < 12
I20250111 09:01:23.915216 16242 tablet_service.cc:1467] Processing CreateTablet for tablet 317a27b34d394490b909c5090de89aaa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=285d52b2a84240f49a1503d8ff3df3f2]), partition=RANGE (txn_id) PARTITION 9 <= VALUES < 12
I20250111 09:01:23.915320 16375 tablet_service.cc:1467] Processing CreateTablet for tablet 317a27b34d394490b909c5090de89aaa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=285d52b2a84240f49a1503d8ff3df3f2]), partition=RANGE (txn_id) PARTITION 9 <= VALUES < 12
I20250111 09:01:23.915441 16104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 317a27b34d394490b909c5090de89aaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:23.915448 16242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 317a27b34d394490b909c5090de89aaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:23.915544 16375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 317a27b34d394490b909c5090de89aaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:23.917265 16538 tablet_bootstrap.cc:492] T 317a27b34d394490b909c5090de89aaa P 91e9fcf6b8ad407dbbf5c8339221b1b8: Bootstrap starting.
I20250111 09:01:23.917559 16540 tablet_bootstrap.cc:492] T 317a27b34d394490b909c5090de89aaa P 2cb0b3b32eb94bf688d9587b86222a0a: Bootstrap starting.
I20250111 09:01:23.918100 16538 tablet_bootstrap.cc:654] T 317a27b34d394490b909c5090de89aaa P 91e9fcf6b8ad407dbbf5c8339221b1b8: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:23.918212 16540 tablet_bootstrap.cc:654] T 317a27b34d394490b909c5090de89aaa P 2cb0b3b32eb94bf688d9587b86222a0a: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:23.918776 16539 tablet_bootstrap.cc:492] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Bootstrap starting.
I20250111 09:01:23.918939 16538 tablet_bootstrap.cc:492] T 317a27b34d394490b909c5090de89aaa P 91e9fcf6b8ad407dbbf5c8339221b1b8: No bootstrap required, opened a new log
I20250111 09:01:23.919014 16538 ts_tablet_manager.cc:1397] T 317a27b34d394490b909c5090de89aaa P 91e9fcf6b8ad407dbbf5c8339221b1b8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:01:23.919291 16538 raft_consensus.cc:357] T 317a27b34d394490b909c5090de89aaa P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.919382 16538 raft_consensus.cc:383] T 317a27b34d394490b909c5090de89aaa P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:23.919415 16538 raft_consensus.cc:738] T 317a27b34d394490b909c5090de89aaa P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e9fcf6b8ad407dbbf5c8339221b1b8, State: Initialized, Role: FOLLOWER
I20250111 09:01:23.919435 16539 tablet_bootstrap.cc:654] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:23.919545 16538 consensus_queue.cc:260] T 317a27b34d394490b909c5090de89aaa P 91e9fcf6b8ad407dbbf5c8339221b1b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.919723 16523 tablet_replica.cc:440] T 317a27b34d394490b909c5090de89aaa P 91e9fcf6b8ad407dbbf5c8339221b1b8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.919791 16538 ts_tablet_manager.cc:1428] T 317a27b34d394490b909c5090de89aaa P 91e9fcf6b8ad407dbbf5c8339221b1b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:01:23.919796 16523 tablet_replica.cc:443] T 317a27b34d394490b909c5090de89aaa P 91e9fcf6b8ad407dbbf5c8339221b1b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.920353 16540 tablet_bootstrap.cc:492] T 317a27b34d394490b909c5090de89aaa P 2cb0b3b32eb94bf688d9587b86222a0a: No bootstrap required, opened a new log
I20250111 09:01:23.920424 16540 ts_tablet_manager.cc:1397] T 317a27b34d394490b909c5090de89aaa P 2cb0b3b32eb94bf688d9587b86222a0a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250111 09:01:23.920897 16540 raft_consensus.cc:357] T 317a27b34d394490b909c5090de89aaa P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.921031 16540 raft_consensus.cc:383] T 317a27b34d394490b909c5090de89aaa P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:23.921092 16540 raft_consensus.cc:738] T 317a27b34d394490b909c5090de89aaa P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cb0b3b32eb94bf688d9587b86222a0a, State: Initialized, Role: FOLLOWER
I20250111 09:01:23.921211 16540 consensus_queue.cc:260] T 317a27b34d394490b909c5090de89aaa P 2cb0b3b32eb94bf688d9587b86222a0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.921406 16470 tablet_replica.cc:440] T 317a27b34d394490b909c5090de89aaa P 2cb0b3b32eb94bf688d9587b86222a0a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.921468 16540 ts_tablet_manager.cc:1428] T 317a27b34d394490b909c5090de89aaa P 2cb0b3b32eb94bf688d9587b86222a0a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:01:23.921473 16470 tablet_replica.cc:443] T 317a27b34d394490b909c5090de89aaa P 2cb0b3b32eb94bf688d9587b86222a0a: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:23.921844 15970 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 556a1f23f24d4b5ba7f0f381d156ced6: kudu_system.kudu_transactions [id=285d52b2a84240f49a1503d8ff3df3f2] alter complete (version 0)
I20250111 09:01:23.941249 16539 tablet_bootstrap.cc:492] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad: No bootstrap required, opened a new log
I20250111 09:01:23.941313 16539 ts_tablet_manager.cc:1397] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.000s
I20250111 09:01:23.941644 16539 raft_consensus.cc:357] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.941735 16539 raft_consensus.cc:383] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:23.941773 16539 raft_consensus.cc:738] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11c4b3d95f0e4ff2b317b15ff32ff0ad, State: Initialized, Role: FOLLOWER
I20250111 09:01:23.941875 16539 consensus_queue.cc:260] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:23.942076 16469 tablet_replica.cc:440] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:23.942119 16539 ts_tablet_manager.cc:1428] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250111 09:01:23.942168 16469 tablet_replica.cc:443] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:24.074299 16469 raft_consensus.cc:491] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:01:24.074426 16469 raft_consensus.cc:513] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:24.074796 16469 leader_election.cc:290] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91e9fcf6b8ad407dbbf5c8339221b1b8 (127.14.61.1:35063), 2cb0b3b32eb94bf688d9587b86222a0a (127.14.61.3:42055)
I20250111 09:01:24.075186 16125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317a27b34d394490b909c5090de89aaa" candidate_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" is_pre_election: true
I20250111 09:01:24.075212 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317a27b34d394490b909c5090de89aaa" candidate_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" is_pre_election: true
I20250111 09:01:24.075327 16125 raft_consensus.cc:2463] T 317a27b34d394490b909c5090de89aaa P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11c4b3d95f0e4ff2b317b15ff32ff0ad in term 0.
I20250111 09:01:24.075348 16399 raft_consensus.cc:2463] T 317a27b34d394490b909c5090de89aaa P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11c4b3d95f0e4ff2b317b15ff32ff0ad in term 0.
I20250111 09:01:24.075671 16193 leader_election.cc:304] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11c4b3d95f0e4ff2b317b15ff32ff0ad, 91e9fcf6b8ad407dbbf5c8339221b1b8; no voters:
I20250111 09:01:24.075848 16469 raft_consensus.cc:2798] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:01:24.075906 16469 raft_consensus.cc:491] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:01:24.075934 16469 raft_consensus.cc:3054] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:24.076709 16469 raft_consensus.cc:513] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:24.076996 16469 leader_election.cc:290] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad [CANDIDATE]: Term 1 election: Requested vote from peers 91e9fcf6b8ad407dbbf5c8339221b1b8 (127.14.61.1:35063), 2cb0b3b32eb94bf688d9587b86222a0a (127.14.61.3:42055)
I20250111 09:01:24.077323 16125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317a27b34d394490b909c5090de89aaa" candidate_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8"
I20250111 09:01:24.077332 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317a27b34d394490b909c5090de89aaa" candidate_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb0b3b32eb94bf688d9587b86222a0a"
I20250111 09:01:24.077435 16399 raft_consensus.cc:3054] T 317a27b34d394490b909c5090de89aaa P 2cb0b3b32eb94bf688d9587b86222a0a [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:24.077435 16125 raft_consensus.cc:3054] T 317a27b34d394490b909c5090de89aaa P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:24.078289 16399 raft_consensus.cc:2463] T 317a27b34d394490b909c5090de89aaa P 2cb0b3b32eb94bf688d9587b86222a0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11c4b3d95f0e4ff2b317b15ff32ff0ad in term 1.
I20250111 09:01:24.078289 16125 raft_consensus.cc:2463] T 317a27b34d394490b909c5090de89aaa P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11c4b3d95f0e4ff2b317b15ff32ff0ad in term 1.
I20250111 09:01:24.078598 16193 leader_election.cc:304] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11c4b3d95f0e4ff2b317b15ff32ff0ad, 91e9fcf6b8ad407dbbf5c8339221b1b8; no voters:
I20250111 09:01:24.078749 16469 raft_consensus.cc:2798] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:24.078853 16469 raft_consensus.cc:695] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad [term 1 LEADER]: Becoming Leader. State: Replica: 11c4b3d95f0e4ff2b317b15ff32ff0ad, State: Running, Role: LEADER
I20250111 09:01:24.078998 16469 consensus_queue.cc:237] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } }
I20250111 09:01:24.079638 16536 tablet_replica.cc:440] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad: TxnStatusTablet state changed. Reason: New leader 11c4b3d95f0e4ff2b317b15ff32ff0ad. Latest consensus state: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:24.079725 16536 tablet_replica.cc:443] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:24.079811 16533 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:01:24.079856 16533 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:01:24.080595 15970 catalog_manager.cc:5526] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 11c4b3d95f0e4ff2b317b15ff32ff0ad (127.14.61.2). New cstate: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:24.096854 16125 raft_consensus.cc:1270] T 317a27b34d394490b909c5090de89aaa P 91e9fcf6b8ad407dbbf5c8339221b1b8 [term 1 FOLLOWER]: Refusing update from remote peer 11c4b3d95f0e4ff2b317b15ff32ff0ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:24.096871 16399 raft_consensus.cc:1270] T 317a27b34d394490b909c5090de89aaa P 2cb0b3b32eb94bf688d9587b86222a0a [term 1 FOLLOWER]: Refusing update from remote peer 11c4b3d95f0e4ff2b317b15ff32ff0ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:24.097209 16469 consensus_queue.cc:1035] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:24.097328 16536 consensus_queue.cc:1035] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:24.098435 16470 tablet_replica.cc:440] T 317a27b34d394490b909c5090de89aaa P 2cb0b3b32eb94bf688d9587b86222a0a: TxnStatusTablet state changed. Reason: New leader 11c4b3d95f0e4ff2b317b15ff32ff0ad. Latest consensus state: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:24.098516 16470 tablet_replica.cc:443] T 317a27b34d394490b909c5090de89aaa P 2cb0b3b32eb94bf688d9587b86222a0a: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:24.098743 16523 tablet_replica.cc:440] T 317a27b34d394490b909c5090de89aaa P 91e9fcf6b8ad407dbbf5c8339221b1b8: TxnStatusTablet state changed. Reason: New leader 11c4b3d95f0e4ff2b317b15ff32ff0ad. Latest consensus state: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:24.098839 16523 tablet_replica.cc:443] T 317a27b34d394490b909c5090de89aaa P 91e9fcf6b8ad407dbbf5c8339221b1b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:24.099704 16509 tablet_replica.cc:440] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:24.099783 16509 tablet_replica.cc:443] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:24.099897 16509 tablet_replica.cc:440] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:24.099975 16509 tablet_replica.cc:443] T 317a27b34d394490b909c5090de89aaa P 11c4b3d95f0e4ff2b317b15ff32ff0ad: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:24.100492 16523 tablet_replica.cc:440] T 317a27b34d394490b909c5090de89aaa P 91e9fcf6b8ad407dbbf5c8339221b1b8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:24.100564 16523 tablet_replica.cc:443] T 317a27b34d394490b909c5090de89aaa P 91e9fcf6b8ad407dbbf5c8339221b1b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:24.101490 16470 tablet_replica.cc:440] T 317a27b34d394490b909c5090de89aaa P 2cb0b3b32eb94bf688d9587b86222a0a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e9fcf6b8ad407dbbf5c8339221b1b8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35063 } } peers { permanent_uuid: "11c4b3d95f0e4ff2b317b15ff32ff0ad" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 37009 } } peers { permanent_uuid: "2cb0b3b32eb94bf688d9587b86222a0a" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 42055 } } }
I20250111 09:01:24.101581 16470 tablet_replica.cc:443] T 317a27b34d394490b909c5090de89aaa P 2cb0b3b32eb94bf688d9587b86222a0a: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:24.133337 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 16041
W20250111 09:01:24.141453 16193 connection.cc:537] client connection to 127.14.61.1:35063 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20250111 09:01:24.141573 16193 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20250111 09:01:24.141945 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 16178
I20250111 09:01:24.147665 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 16315
I20250111 09:01:24.152606 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 15940
2025-01-11T09:01:24Z chronyd exiting
[ OK ] TxnWriteOpsITest.TestWriteToNewRangeOfTxnIds (2053 ms)
[ RUN ] TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit
2025-01-11T09:01:24Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-01-11T09:01:24Z Disabled control of system clock
I20250111 09:01:24.181392 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.61.62:45983
--webserver_interface=127.14.61.62
--webserver_port=0
--builtin_ntp_servers=127.14.61.20:34097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.61.62:45983
--txn_manager_enabled=true
--txn_manager_lazily_initialized=false with env {}
W20250111 09:01:24.272193 16552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:01:24.272358 16552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:01:24.272392 16552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:01:24.275228 16552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250111 09:01:24.275272 16552 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20250111 09:01:24.275287 16552 flags.cc:425] Enabled experimental flag: --txn_manager_lazily_initialized=false
W20250111 09:01:24.275301 16552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:01:24.275321 16552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250111 09:01:24.275337 16552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250111 09:01:24.278415 16552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:34097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.61.62:45983
--txn_manager_enabled=true
--txn_manager_lazily_initialized=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.61.62:45983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.61.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.16552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:01:24.279176 16552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:01:24.280051 16552 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:01:24.286288 16560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:24.286898 16557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:24.287133 16558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:01:24.287853 16552 server_base.cc:1034] running on GCE node
I20250111 09:01:24.288242 16552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:01:24.288969 16552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:01:24.290128 16552 hybrid_clock.cc:648] HybridClock initialized: now 1736586084290115 us; error 25 us; skew 500 ppm
I20250111 09:01:24.291762 16552 webserver.cc:458] Webserver started at http://127.14.61.62:39683/ using document root <none> and password file <none>
I20250111 09:01:24.292207 16552 fs_manager.cc:362] Metadata directory not provided
I20250111 09:01:24.292258 16552 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:01:24.292394 16552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:01:24.293859 16552 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/master-0/data/instance:
uuid: "a866661d39f9467ca0e128f545b94388"
format_stamp: "Formatted at 2025-01-11 09:01:24 on dist-test-slave-nvbt"
I20250111 09:01:24.294291 16552 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/master-0/wal/instance:
uuid: "a866661d39f9467ca0e128f545b94388"
format_stamp: "Formatted at 2025-01-11 09:01:24 on dist-test-slave-nvbt"
I20250111 09:01:24.296988 16552 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20250111 09:01:24.298694 16566 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:24.299614 16552 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20250111 09:01:24.299728 16552 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/master-0/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/master-0/wal
uuid: "a866661d39f9467ca0e128f545b94388"
format_stamp: "Formatted at 2025-01-11 09:01:24 on dist-test-slave-nvbt"
I20250111 09:01:24.299795 16552 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:01:24.314977 16552 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:01:24.315479 16552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:01:24.315629 16552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:01:24.322048 16552 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:45983
I20250111 09:01:24.322045 16628 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:45983 every 8 connection(s)
I20250111 09:01:24.322919 16552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
I20250111 09:01:24.325110 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 16552
I20250111 09:01:24.325228 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/master-0/wal/instance
I20250111 09:01:24.325282 16629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:24.330185 16629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388: Bootstrap starting.
I20250111 09:01:24.332326 16629 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:24.333007 16629 log.cc:826] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388: Log is configured to *not* fsync() on all Append() calls
I20250111 09:01:24.334503 16629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388: No bootstrap required, opened a new log
I20250111 09:01:24.337266 16629 raft_consensus.cc:357] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a866661d39f9467ca0e128f545b94388" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 45983 } }
I20250111 09:01:24.337410 16629 raft_consensus.cc:383] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:24.337456 16629 raft_consensus.cc:738] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a866661d39f9467ca0e128f545b94388, State: Initialized, Role: FOLLOWER
I20250111 09:01:24.337829 16629 consensus_queue.cc:260] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a866661d39f9467ca0e128f545b94388" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 45983 } }
I20250111 09:01:24.337934 16629 raft_consensus.cc:397] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:01:24.337989 16629 raft_consensus.cc:491] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:01:24.338088 16629 raft_consensus.cc:3054] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:24.338666 16629 raft_consensus.cc:513] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a866661d39f9467ca0e128f545b94388" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 45983 } }
I20250111 09:01:24.338950 16629 leader_election.cc:304] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a866661d39f9467ca0e128f545b94388; no voters:
I20250111 09:01:24.339224 16629 leader_election.cc:290] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:01:24.339326 16634 raft_consensus.cc:2798] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:24.339527 16634 raft_consensus.cc:695] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388 [term 1 LEADER]: Becoming Leader. State: Replica: a866661d39f9467ca0e128f545b94388, State: Running, Role: LEADER
I20250111 09:01:24.339854 16634 consensus_queue.cc:237] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a866661d39f9467ca0e128f545b94388" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 45983 } }
I20250111 09:01:24.340113 16629 sys_catalog.cc:564] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388 [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:01:24.341189 16636 sys_catalog.cc:455] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a866661d39f9467ca0e128f545b94388. Latest consensus state: current_term: 1 leader_uuid: "a866661d39f9467ca0e128f545b94388" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a866661d39f9467ca0e128f545b94388" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 45983 } } }
I20250111 09:01:24.341289 16636 sys_catalog.cc:458] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388 [sys.catalog]: This master's current role is: LEADER
I20250111 09:01:24.341619 16635 sys_catalog.cc:455] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a866661d39f9467ca0e128f545b94388" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a866661d39f9467ca0e128f545b94388" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 45983 } } }
I20250111 09:01:24.341658 16643 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:01:24.341774 16635 sys_catalog.cc:458] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388 [sys.catalog]: This master's current role is: LEADER
I20250111 09:01:24.344651 16643 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:01:24.349525 16643 catalog_manager.cc:1348] Generated new cluster ID: 81d6a7c11ae54905aa32099da78d985d
I20250111 09:01:24.349582 16643 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:01:24.359968 16643 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:01:24.360729 16643 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:01:24.374492 16643 catalog_manager.cc:5899] T 00000000000000000000000000000000 P a866661d39f9467ca0e128f545b94388: Generated new TSK 0
I20250111 09:01:24.375092 16643 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:01:24.380044 16583 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55280:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:01:24.383210 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.1:0
--local_ip_for_outbound_sockets=127.14.61.1
--webserver_interface=127.14.61.1
--webserver_port=0
--tserver_master_addrs=127.14.61.62:45983
--builtin_ntp_servers=127.14.61.20:34097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_txn_system_client_init=true with env {}
W20250111 09:01:24.383890 16629 master.cc:440] Invalid argument: unable to initialize TxnManager: Error creating table kudu_system.kudu_transactions on the master: not enough live tablet servers to create a table with the requested replication factor 3; 0 tablet servers are alive: unable to init TxnManager, will retry in 1.000s
W20250111 09:01:24.476827 16658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:01:24.477118 16658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:01:24.477190 16658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:01:24.480047 16658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:01:24.480110 16658 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:01:24.480152 16658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.1
I20250111 09:01:24.483436 16658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:34097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.61.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:45983
--never_fsync=true
--heap_profile_path=/tmp/kudu.16658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.1
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:01:24.484377 16658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:01:24.485360 16658 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:01:24.492117 16663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:24.492298 16666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:24.492898 16664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:01:24.493609 16658 server_base.cc:1034] running on GCE node
I20250111 09:01:24.493896 16658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:01:24.494324 16658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:01:24.495489 16658 hybrid_clock.cc:648] HybridClock initialized: now 1736586084495468 us; error 34 us; skew 500 ppm
I20250111 09:01:24.497062 16658 webserver.cc:458] Webserver started at http://127.14.61.1:34333/ using document root <none> and password file <none>
I20250111 09:01:24.497529 16658 fs_manager.cc:362] Metadata directory not provided
I20250111 09:01:24.497577 16658 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:01:24.497747 16658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:01:24.499126 16658 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-0/data/instance:
uuid: "e7bb11a869a542929782a67bbc8a72d8"
format_stamp: "Formatted at 2025-01-11 09:01:24 on dist-test-slave-nvbt"
I20250111 09:01:24.499531 16658 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-0/wal/instance:
uuid: "e7bb11a869a542929782a67bbc8a72d8"
format_stamp: "Formatted at 2025-01-11 09:01:24 on dist-test-slave-nvbt"
I20250111 09:01:24.502429 16658 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20250111 09:01:24.504329 16672 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:24.505250 16658 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250111 09:01:24.505348 16658 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-0/wal
uuid: "e7bb11a869a542929782a67bbc8a72d8"
format_stamp: "Formatted at 2025-01-11 09:01:24 on dist-test-slave-nvbt"
I20250111 09:01:24.505424 16658 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:01:24.518536 16658 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:01:24.518954 16658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:01:24.519079 16658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:01:24.520232 16658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:01:24.520280 16658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:24.520329 16658 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:01:24.520359 16658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:24.529827 16658 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:38471
I20250111 09:01:24.529868 16792 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:38471 every 8 connection(s)
I20250111 09:01:24.530546 16658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
I20250111 09:01:24.531666 16793 heartbeater.cc:346] Connected to a master server at 127.14.61.62:45983
I20250111 09:01:24.531773 16793 heartbeater.cc:463] Registering TS with master...
I20250111 09:01:24.532198 16793 heartbeater.cc:510] Master 127.14.61.62:45983 requested a full tablet report, sending...
I20250111 09:01:24.533308 16583 ts_manager.cc:194] Registered new tserver with Master: e7bb11a869a542929782a67bbc8a72d8 (127.14.61.1:38471)
I20250111 09:01:24.534554 16583 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.1:57633
I20250111 09:01:24.539324 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 16658
I20250111 09:01:24.539408 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-0/wal/instance
I20250111 09:01:24.540968 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.2:0
--local_ip_for_outbound_sockets=127.14.61.2
--webserver_interface=127.14.61.2
--webserver_port=0
--tserver_master_addrs=127.14.61.62:45983
--builtin_ntp_servers=127.14.61.20:34097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_txn_system_client_init=true with env {}
W20250111 09:01:24.635056 16795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:01:24.635231 16795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:01:24.635310 16795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:01:24.638042 16795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:01:24.638132 16795 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:01:24.638212 16795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.2
I20250111 09:01:24.641389 16795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:34097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.61.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:45983
--never_fsync=true
--heap_profile_path=/tmp/kudu.16795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.2
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:01:24.642171 16795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:01:24.643144 16795 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:01:24.649698 16803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:24.649869 16800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:24.650477 16801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:01:24.651137 16795 server_base.cc:1034] running on GCE node
I20250111 09:01:24.651400 16795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:01:24.651856 16795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:01:24.653043 16795 hybrid_clock.cc:648] HybridClock initialized: now 1736586084653002 us; error 30 us; skew 500 ppm
I20250111 09:01:24.654558 16795 webserver.cc:458] Webserver started at http://127.14.61.2:44699/ using document root <none> and password file <none>
I20250111 09:01:24.655016 16795 fs_manager.cc:362] Metadata directory not provided
I20250111 09:01:24.655067 16795 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:01:24.655232 16795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:01:24.656554 16795 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-1/data/instance:
uuid: "d5cf50b8a8b940629e21ab677ea44e45"
format_stamp: "Formatted at 2025-01-11 09:01:24 on dist-test-slave-nvbt"
I20250111 09:01:24.656903 16795 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-1/wal/instance:
uuid: "d5cf50b8a8b940629e21ab677ea44e45"
format_stamp: "Formatted at 2025-01-11 09:01:24 on dist-test-slave-nvbt"
I20250111 09:01:24.659593 16795 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20250111 09:01:24.661165 16809 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:24.662082 16795 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:01:24.662184 16795 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-1/wal
uuid: "d5cf50b8a8b940629e21ab677ea44e45"
format_stamp: "Formatted at 2025-01-11 09:01:24 on dist-test-slave-nvbt"
I20250111 09:01:24.662269 16795 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:01:24.685928 16795 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:01:24.686314 16795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:01:24.686450 16795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:01:24.687536 16795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:01:24.687598 16795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:24.687637 16795 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:01:24.687662 16795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:24.696113 16795 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.2:44661
I20250111 09:01:24.696146 16928 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.2:44661 every 8 connection(s)
I20250111 09:01:24.696916 16795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
I20250111 09:01:24.697930 16930 heartbeater.cc:346] Connected to a master server at 127.14.61.62:45983
I20250111 09:01:24.698028 16930 heartbeater.cc:463] Registering TS with master...
I20250111 09:01:24.698334 16930 heartbeater.cc:510] Master 127.14.61.62:45983 requested a full tablet report, sending...
I20250111 09:01:24.699447 16583 ts_manager.cc:194] Registered new tserver with Master: d5cf50b8a8b940629e21ab677ea44e45 (127.14.61.2:44661)
I20250111 09:01:24.700042 16583 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.2:39153
I20250111 09:01:24.705863 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 16795
I20250111 09:01:24.705962 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-1/wal/instance
I20250111 09:01:24.707587 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.3:0
--local_ip_for_outbound_sockets=127.14.61.3
--webserver_interface=127.14.61.3
--webserver_port=0
--tserver_master_addrs=127.14.61.62:45983
--builtin_ntp_servers=127.14.61.20:34097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_txn_system_client_init=true with env {}
W20250111 09:01:24.800060 16932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:01:24.800256 16932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:01:24.800323 16932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:01:24.803164 16932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:01:24.803252 16932 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:01:24.803337 16932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.3
I20250111 09:01:24.806695 16932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:34097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.61.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:45983
--never_fsync=true
--heap_profile_path=/tmp/kudu.16932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.3
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:01:24.807440 16932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:01:24.808270 16932 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:01:24.815635 16938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:24.815757 16937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:24.816382 16940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:01:24.817081 16932 server_base.cc:1034] running on GCE node
I20250111 09:01:24.817339 16932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:01:24.817735 16932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:01:24.818873 16932 hybrid_clock.cc:648] HybridClock initialized: now 1736586084818855 us; error 37 us; skew 500 ppm
I20250111 09:01:24.820458 16932 webserver.cc:458] Webserver started at http://127.14.61.3:38631/ using document root <none> and password file <none>
I20250111 09:01:24.820883 16932 fs_manager.cc:362] Metadata directory not provided
I20250111 09:01:24.820935 16932 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:01:24.821147 16932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:01:24.822585 16932 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-2/data/instance:
uuid: "f744ff3c16b04568a8a771904eda03cb"
format_stamp: "Formatted at 2025-01-11 09:01:24 on dist-test-slave-nvbt"
I20250111 09:01:24.823004 16932 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-2/wal/instance:
uuid: "f744ff3c16b04568a8a771904eda03cb"
format_stamp: "Formatted at 2025-01-11 09:01:24 on dist-test-slave-nvbt"
I20250111 09:01:24.826050 16932 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20250111 09:01:24.827816 16946 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:24.828725 16932 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:01:24.828814 16932 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-2/wal
uuid: "f744ff3c16b04568a8a771904eda03cb"
format_stamp: "Formatted at 2025-01-11 09:01:24 on dist-test-slave-nvbt"
I20250111 09:01:24.828881 16932 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:01:24.847169 16932 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:01:24.847577 16932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:01:24.847713 16932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:01:24.848783 16932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:01:24.848827 16932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:24.848871 16932 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:01:24.848899 16932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:24.857863 16932 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.3:35667
I20250111 09:01:24.857931 17061 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.3:35667 every 8 connection(s)
I20250111 09:01:24.858836 16932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
I20250111 09:01:24.860229 17066 heartbeater.cc:346] Connected to a master server at 127.14.61.62:45983
I20250111 09:01:24.860339 17066 heartbeater.cc:463] Registering TS with master...
I20250111 09:01:24.860728 17066 heartbeater.cc:510] Master 127.14.61.62:45983 requested a full tablet report, sending...
I20250111 09:01:24.861557 16583 ts_manager.cc:194] Registered new tserver with Master: f744ff3c16b04568a8a771904eda03cb (127.14.61.3:35667)
I20250111 09:01:24.862169 16583 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.3:55729
I20250111 09:01:24.862540 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 16932
I20250111 09:01:24.862650 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit.1736586070622223-14580-0/minicluster-data/ts-2/wal/instance
I20250111 09:01:24.864650 14580 external_mini_cluster.cc:934] 3 TS(s) registered with all masters
I20250111 09:01:24.873117 16582 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51100:
name: "txn_write_ops_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
W20250111 09:01:24.873706 16582 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table txn_write_ops_test in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250111 09:01:24.890801 16996 tablet_service.cc:1467] Processing CreateTablet for tablet e5ba02abf0ab41bb8dd3ec6a4e28422c (DEFAULT_TABLE table=txn_write_ops_test [id=a04cf5c2ff7b4a6eb5512ea91dfa9d07]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:24.890807 16995 tablet_service.cc:1467] Processing CreateTablet for tablet 358e989064ec4d7b8708607b6a0da029 (DEFAULT_TABLE table=txn_write_ops_test [id=a04cf5c2ff7b4a6eb5512ea91dfa9d07]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:24.891697 16996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5ba02abf0ab41bb8dd3ec6a4e28422c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:24.891712 16859 tablet_service.cc:1467] Processing CreateTablet for tablet e5ba02abf0ab41bb8dd3ec6a4e28422c (DEFAULT_TABLE table=txn_write_ops_test [id=a04cf5c2ff7b4a6eb5512ea91dfa9d07]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:24.892031 16995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 358e989064ec4d7b8708607b6a0da029. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:24.892482 16859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5ba02abf0ab41bb8dd3ec6a4e28422c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:24.893277 16858 tablet_service.cc:1467] Processing CreateTablet for tablet 358e989064ec4d7b8708607b6a0da029 (DEFAULT_TABLE table=txn_write_ops_test [id=a04cf5c2ff7b4a6eb5512ea91dfa9d07]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:24.893522 16858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 358e989064ec4d7b8708607b6a0da029. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:24.896333 16722 tablet_service.cc:1467] Processing CreateTablet for tablet e5ba02abf0ab41bb8dd3ec6a4e28422c (DEFAULT_TABLE table=txn_write_ops_test [id=a04cf5c2ff7b4a6eb5512ea91dfa9d07]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:24.897154 16722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5ba02abf0ab41bb8dd3ec6a4e28422c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:24.898597 16721 tablet_service.cc:1467] Processing CreateTablet for tablet 358e989064ec4d7b8708607b6a0da029 (DEFAULT_TABLE table=txn_write_ops_test [id=a04cf5c2ff7b4a6eb5512ea91dfa9d07]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:24.898882 16721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 358e989064ec4d7b8708607b6a0da029. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:24.899924 17084 tablet_bootstrap.cc:492] T e5ba02abf0ab41bb8dd3ec6a4e28422c P d5cf50b8a8b940629e21ab677ea44e45: Bootstrap starting.
I20250111 09:01:24.901237 17083 tablet_bootstrap.cc:492] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb: Bootstrap starting.
I20250111 09:01:24.902168 17084 tablet_bootstrap.cc:654] T e5ba02abf0ab41bb8dd3ec6a4e28422c P d5cf50b8a8b940629e21ab677ea44e45: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:24.903012 17084 log.cc:826] T e5ba02abf0ab41bb8dd3ec6a4e28422c P d5cf50b8a8b940629e21ab677ea44e45: Log is configured to *not* fsync() on all Append() calls
I20250111 09:01:24.904189 17083 tablet_bootstrap.cc:654] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:24.904542 17084 tablet_bootstrap.cc:492] T e5ba02abf0ab41bb8dd3ec6a4e28422c P d5cf50b8a8b940629e21ab677ea44e45: No bootstrap required, opened a new log
I20250111 09:01:24.904748 17084 ts_tablet_manager.cc:1397] T e5ba02abf0ab41bb8dd3ec6a4e28422c P d5cf50b8a8b940629e21ab677ea44e45: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20250111 09:01:24.904886 17083 log.cc:826] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb: Log is configured to *not* fsync() on all Append() calls
I20250111 09:01:24.906345 17083 tablet_bootstrap.cc:492] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb: No bootstrap required, opened a new log
I20250111 09:01:24.906512 17083 ts_tablet_manager.cc:1397] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20250111 09:01:24.907683 17084 raft_consensus.cc:357] T e5ba02abf0ab41bb8dd3ec6a4e28422c P d5cf50b8a8b940629e21ab677ea44e45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } }
I20250111 09:01:24.907863 17084 raft_consensus.cc:383] T e5ba02abf0ab41bb8dd3ec6a4e28422c P d5cf50b8a8b940629e21ab677ea44e45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:24.907912 17084 raft_consensus.cc:738] T e5ba02abf0ab41bb8dd3ec6a4e28422c P d5cf50b8a8b940629e21ab677ea44e45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5cf50b8a8b940629e21ab677ea44e45, State: Initialized, Role: FOLLOWER
I20250111 09:01:24.908325 17084 consensus_queue.cc:260] T e5ba02abf0ab41bb8dd3ec6a4e28422c P d5cf50b8a8b940629e21ab677ea44e45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } }
I20250111 09:01:24.908919 17087 tablet_bootstrap.cc:492] T 358e989064ec4d7b8708607b6a0da029 P e7bb11a869a542929782a67bbc8a72d8: Bootstrap starting.
I20250111 09:01:24.909138 17084 ts_tablet_manager.cc:1428] T e5ba02abf0ab41bb8dd3ec6a4e28422c P d5cf50b8a8b940629e21ab677ea44e45: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20250111 09:01:24.909255 17083 raft_consensus.cc:357] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } }
I20250111 09:01:24.909422 17083 raft_consensus.cc:383] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:24.909437 16930 heartbeater.cc:502] Master 127.14.61.62:45983 was elected leader, sending a full tablet report...
I20250111 09:01:24.909480 17083 raft_consensus.cc:738] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f744ff3c16b04568a8a771904eda03cb, State: Initialized, Role: FOLLOWER
I20250111 09:01:24.910437 17083 consensus_queue.cc:260] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } }
I20250111 09:01:24.911121 17083 ts_tablet_manager.cc:1428] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20250111 09:01:24.911440 17083 tablet_bootstrap.cc:492] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb: Bootstrap starting.
I20250111 09:01:24.911993 17066 heartbeater.cc:502] Master 127.14.61.62:45983 was elected leader, sending a full tablet report...
I20250111 09:01:24.912087 17087 tablet_bootstrap.cc:654] T 358e989064ec4d7b8708607b6a0da029 P e7bb11a869a542929782a67bbc8a72d8: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:24.912328 17083 tablet_bootstrap.cc:654] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:24.913213 17087 log.cc:826] T 358e989064ec4d7b8708607b6a0da029 P e7bb11a869a542929782a67bbc8a72d8: Log is configured to *not* fsync() on all Append() calls
I20250111 09:01:24.913390 17084 tablet_bootstrap.cc:492] T 358e989064ec4d7b8708607b6a0da029 P d5cf50b8a8b940629e21ab677ea44e45: Bootstrap starting.
I20250111 09:01:24.914433 17084 tablet_bootstrap.cc:654] T 358e989064ec4d7b8708607b6a0da029 P d5cf50b8a8b940629e21ab677ea44e45: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:24.914965 17087 tablet_bootstrap.cc:492] T 358e989064ec4d7b8708607b6a0da029 P e7bb11a869a542929782a67bbc8a72d8: No bootstrap required, opened a new log
I20250111 09:01:24.915117 17087 ts_tablet_manager.cc:1397] T 358e989064ec4d7b8708607b6a0da029 P e7bb11a869a542929782a67bbc8a72d8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20250111 09:01:24.916326 17084 tablet_bootstrap.cc:492] T 358e989064ec4d7b8708607b6a0da029 P d5cf50b8a8b940629e21ab677ea44e45: No bootstrap required, opened a new log
I20250111 09:01:24.916365 17083 tablet_bootstrap.cc:492] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb: No bootstrap required, opened a new log
I20250111 09:01:24.916397 17084 ts_tablet_manager.cc:1397] T 358e989064ec4d7b8708607b6a0da029 P d5cf50b8a8b940629e21ab677ea44e45: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250111 09:01:24.916420 17083 ts_tablet_manager.cc:1397] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20250111 09:01:24.916767 17084 raft_consensus.cc:357] T 358e989064ec4d7b8708607b6a0da029 P d5cf50b8a8b940629e21ab677ea44e45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } }
I20250111 09:01:24.916755 17083 raft_consensus.cc:357] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } }
I20250111 09:01:24.916838 17083 raft_consensus.cc:383] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:24.916838 17084 raft_consensus.cc:383] T 358e989064ec4d7b8708607b6a0da029 P d5cf50b8a8b940629e21ab677ea44e45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:24.916867 17083 raft_consensus.cc:738] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f744ff3c16b04568a8a771904eda03cb, State: Initialized, Role: FOLLOWER
I20250111 09:01:24.916867 17084 raft_consensus.cc:738] T 358e989064ec4d7b8708607b6a0da029 P d5cf50b8a8b940629e21ab677ea44e45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5cf50b8a8b940629e21ab677ea44e45, State: Initialized, Role: FOLLOWER
I20250111 09:01:24.916965 17084 consensus_queue.cc:260] T 358e989064ec4d7b8708607b6a0da029 P d5cf50b8a8b940629e21ab677ea44e45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } }
I20250111 09:01:24.917208 17084 ts_tablet_manager.cc:1428] T 358e989064ec4d7b8708607b6a0da029 P d5cf50b8a8b940629e21ab677ea44e45: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:01:24.916965 17083 consensus_queue.cc:260] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } }
I20250111 09:01:24.917543 17083 ts_tablet_manager.cc:1428] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:01:24.918644 17087 raft_consensus.cc:357] T 358e989064ec4d7b8708607b6a0da029 P e7bb11a869a542929782a67bbc8a72d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } }
I20250111 09:01:24.918853 17087 raft_consensus.cc:383] T 358e989064ec4d7b8708607b6a0da029 P e7bb11a869a542929782a67bbc8a72d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:24.918921 17087 raft_consensus.cc:738] T 358e989064ec4d7b8708607b6a0da029 P e7bb11a869a542929782a67bbc8a72d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7bb11a869a542929782a67bbc8a72d8, State: Initialized, Role: FOLLOWER
I20250111 09:01:24.919409 17087 consensus_queue.cc:260] T 358e989064ec4d7b8708607b6a0da029 P e7bb11a869a542929782a67bbc8a72d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } }
I20250111 09:01:24.920275 17087 ts_tablet_manager.cc:1428] T 358e989064ec4d7b8708607b6a0da029 P e7bb11a869a542929782a67bbc8a72d8: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20250111 09:01:24.920455 16793 heartbeater.cc:502] Master 127.14.61.62:45983 was elected leader, sending a full tablet report...
I20250111 09:01:24.920519 17087 tablet_bootstrap.cc:492] T e5ba02abf0ab41bb8dd3ec6a4e28422c P e7bb11a869a542929782a67bbc8a72d8: Bootstrap starting.
I20250111 09:01:24.921795 17087 tablet_bootstrap.cc:654] T e5ba02abf0ab41bb8dd3ec6a4e28422c P e7bb11a869a542929782a67bbc8a72d8: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:24.922978 17087 tablet_bootstrap.cc:492] T e5ba02abf0ab41bb8dd3ec6a4e28422c P e7bb11a869a542929782a67bbc8a72d8: No bootstrap required, opened a new log
I20250111 09:01:24.923067 17087 ts_tablet_manager.cc:1397] T e5ba02abf0ab41bb8dd3ec6a4e28422c P e7bb11a869a542929782a67bbc8a72d8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250111 09:01:24.923377 17087 raft_consensus.cc:357] T e5ba02abf0ab41bb8dd3ec6a4e28422c P e7bb11a869a542929782a67bbc8a72d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } }
I20250111 09:01:24.923455 17087 raft_consensus.cc:383] T e5ba02abf0ab41bb8dd3ec6a4e28422c P e7bb11a869a542929782a67bbc8a72d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:24.923477 17087 raft_consensus.cc:738] T e5ba02abf0ab41bb8dd3ec6a4e28422c P e7bb11a869a542929782a67bbc8a72d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7bb11a869a542929782a67bbc8a72d8, State: Initialized, Role: FOLLOWER
I20250111 09:01:24.923592 17087 consensus_queue.cc:260] T e5ba02abf0ab41bb8dd3ec6a4e28422c P e7bb11a869a542929782a67bbc8a72d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } }
I20250111 09:01:24.923818 17087 ts_tablet_manager.cc:1428] T e5ba02abf0ab41bb8dd3ec6a4e28422c P e7bb11a869a542929782a67bbc8a72d8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20250111 09:01:24.947916 16931 tablet.cc:2367] T 358e989064ec4d7b8708607b6a0da029 P d5cf50b8a8b940629e21ab677ea44e45: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250111 09:01:24.991561 17089 raft_consensus.cc:491] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:01:24.991681 17089 raft_consensus.cc:513] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } }
I20250111 09:01:24.992596 17089 leader_election.cc:290] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7bb11a869a542929782a67bbc8a72d8 (127.14.61.1:38471), d5cf50b8a8b940629e21ab677ea44e45 (127.14.61.2:44661)
I20250111 09:01:24.996338 16879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5ba02abf0ab41bb8dd3ec6a4e28422c" candidate_uuid: "f744ff3c16b04568a8a771904eda03cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5cf50b8a8b940629e21ab677ea44e45" is_pre_election: true
I20250111 09:01:24.996486 16742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5ba02abf0ab41bb8dd3ec6a4e28422c" candidate_uuid: "f744ff3c16b04568a8a771904eda03cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7bb11a869a542929782a67bbc8a72d8" is_pre_election: true
I20250111 09:01:24.996560 16879 raft_consensus.cc:2463] T e5ba02abf0ab41bb8dd3ec6a4e28422c P d5cf50b8a8b940629e21ab677ea44e45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f744ff3c16b04568a8a771904eda03cb in term 0.
I20250111 09:01:24.996697 16742 raft_consensus.cc:2463] T e5ba02abf0ab41bb8dd3ec6a4e28422c P e7bb11a869a542929782a67bbc8a72d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f744ff3c16b04568a8a771904eda03cb in term 0.
I20250111 09:01:24.996946 16950 leader_election.cc:304] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5cf50b8a8b940629e21ab677ea44e45, f744ff3c16b04568a8a771904eda03cb; no voters:
I20250111 09:01:24.997164 17089 raft_consensus.cc:2798] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:01:24.997231 17089 raft_consensus.cc:491] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:01:24.997284 17089 raft_consensus.cc:3054] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:24.998023 17089 raft_consensus.cc:513] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } }
I20250111 09:01:24.998286 17089 leader_election.cc:290] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb [CANDIDATE]: Term 1 election: Requested vote from peers e7bb11a869a542929782a67bbc8a72d8 (127.14.61.1:38471), d5cf50b8a8b940629e21ab677ea44e45 (127.14.61.2:44661)
I20250111 09:01:24.998659 16879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5ba02abf0ab41bb8dd3ec6a4e28422c" candidate_uuid: "f744ff3c16b04568a8a771904eda03cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5cf50b8a8b940629e21ab677ea44e45"
I20250111 09:01:24.998679 16742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5ba02abf0ab41bb8dd3ec6a4e28422c" candidate_uuid: "f744ff3c16b04568a8a771904eda03cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7bb11a869a542929782a67bbc8a72d8"
I20250111 09:01:24.998759 16879 raft_consensus.cc:3054] T e5ba02abf0ab41bb8dd3ec6a4e28422c P d5cf50b8a8b940629e21ab677ea44e45 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:24.998759 16742 raft_consensus.cc:3054] T e5ba02abf0ab41bb8dd3ec6a4e28422c P e7bb11a869a542929782a67bbc8a72d8 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:24.999505 16742 raft_consensus.cc:2463] T e5ba02abf0ab41bb8dd3ec6a4e28422c P e7bb11a869a542929782a67bbc8a72d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f744ff3c16b04568a8a771904eda03cb in term 1.
I20250111 09:01:24.999565 16879 raft_consensus.cc:2463] T e5ba02abf0ab41bb8dd3ec6a4e28422c P d5cf50b8a8b940629e21ab677ea44e45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f744ff3c16b04568a8a771904eda03cb in term 1.
I20250111 09:01:24.999771 16950 leader_election.cc:304] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7bb11a869a542929782a67bbc8a72d8, f744ff3c16b04568a8a771904eda03cb; no voters:
I20250111 09:01:24.999922 17089 raft_consensus.cc:2798] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:25.000098 17089 raft_consensus.cc:695] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb [term 1 LEADER]: Becoming Leader. State: Replica: f744ff3c16b04568a8a771904eda03cb, State: Running, Role: LEADER
I20250111 09:01:25.000329 17089 consensus_queue.cc:237] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } }
I20250111 09:01:25.002841 16581 catalog_manager.cc:5526] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb reported cstate change: term changed from 0 to 1, leader changed from <none> to f744ff3c16b04568a8a771904eda03cb (127.14.61.3). New cstate: current_term: 1 leader_uuid: "f744ff3c16b04568a8a771904eda03cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:25.031492 16794 tablet.cc:2367] T 358e989064ec4d7b8708607b6a0da029 P e7bb11a869a542929782a67bbc8a72d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250111 09:01:25.075349 17089 raft_consensus.cc:491] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:01:25.075531 17089 raft_consensus.cc:513] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } }
I20250111 09:01:25.075822 17089 leader_election.cc:290] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7bb11a869a542929782a67bbc8a72d8 (127.14.61.1:38471), d5cf50b8a8b940629e21ab677ea44e45 (127.14.61.2:44661)
I20250111 09:01:25.076100 16742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "358e989064ec4d7b8708607b6a0da029" candidate_uuid: "f744ff3c16b04568a8a771904eda03cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7bb11a869a542929782a67bbc8a72d8" is_pre_election: true
I20250111 09:01:25.076189 16742 raft_consensus.cc:2463] T 358e989064ec4d7b8708607b6a0da029 P e7bb11a869a542929782a67bbc8a72d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f744ff3c16b04568a8a771904eda03cb in term 0.
I20250111 09:01:25.076180 16879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "358e989064ec4d7b8708607b6a0da029" candidate_uuid: "f744ff3c16b04568a8a771904eda03cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5cf50b8a8b940629e21ab677ea44e45" is_pre_election: true
I20250111 09:01:25.076289 16879 raft_consensus.cc:2463] T 358e989064ec4d7b8708607b6a0da029 P d5cf50b8a8b940629e21ab677ea44e45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f744ff3c16b04568a8a771904eda03cb in term 0.
I20250111 09:01:25.076417 16950 leader_election.cc:304] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7bb11a869a542929782a67bbc8a72d8, f744ff3c16b04568a8a771904eda03cb; no voters:
I20250111 09:01:25.076565 17089 raft_consensus.cc:2798] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:01:25.076622 17089 raft_consensus.cc:491] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:01:25.076645 17089 raft_consensus.cc:3054] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:25.077271 17089 raft_consensus.cc:513] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } }
I20250111 09:01:25.077504 17089 leader_election.cc:290] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb [CANDIDATE]: Term 1 election: Requested vote from peers e7bb11a869a542929782a67bbc8a72d8 (127.14.61.1:38471), d5cf50b8a8b940629e21ab677ea44e45 (127.14.61.2:44661)
I20250111 09:01:25.077765 16742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "358e989064ec4d7b8708607b6a0da029" candidate_uuid: "f744ff3c16b04568a8a771904eda03cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7bb11a869a542929782a67bbc8a72d8"
I20250111 09:01:25.077836 16879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "358e989064ec4d7b8708607b6a0da029" candidate_uuid: "f744ff3c16b04568a8a771904eda03cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5cf50b8a8b940629e21ab677ea44e45"
I20250111 09:01:25.077862 16742 raft_consensus.cc:3054] T 358e989064ec4d7b8708607b6a0da029 P e7bb11a869a542929782a67bbc8a72d8 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:25.077934 16879 raft_consensus.cc:3054] T 358e989064ec4d7b8708607b6a0da029 P d5cf50b8a8b940629e21ab677ea44e45 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:25.078511 16742 raft_consensus.cc:2463] T 358e989064ec4d7b8708607b6a0da029 P e7bb11a869a542929782a67bbc8a72d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f744ff3c16b04568a8a771904eda03cb in term 1.
I20250111 09:01:25.078539 16879 raft_consensus.cc:2463] T 358e989064ec4d7b8708607b6a0da029 P d5cf50b8a8b940629e21ab677ea44e45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f744ff3c16b04568a8a771904eda03cb in term 1.
I20250111 09:01:25.078776 16950 leader_election.cc:304] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5cf50b8a8b940629e21ab677ea44e45, f744ff3c16b04568a8a771904eda03cb; no voters:
I20250111 09:01:25.078931 17089 raft_consensus.cc:2798] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:25.079013 17089 raft_consensus.cc:695] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb [term 1 LEADER]: Becoming Leader. State: Replica: f744ff3c16b04568a8a771904eda03cb, State: Running, Role: LEADER
I20250111 09:01:25.079134 17089 consensus_queue.cc:237] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } }
I20250111 09:01:25.080364 16581 catalog_manager.cc:5526] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb reported cstate change: term changed from 0 to 1, leader changed from <none> to f744ff3c16b04568a8a771904eda03cb (127.14.61.3). New cstate: current_term: 1 leader_uuid: "f744ff3c16b04568a8a771904eda03cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } health_report { overall_health: HEALTHY } } }
W20250111 09:01:25.115561 17067 tablet.cc:2367] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250111 09:01:25.131825 17069 txn_manager_proxy_rpc.cc:150] re-attempting BeginTransaction request to one of TxnManagers
I20250111 09:01:25.397370 16581 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51104:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20250111 09:01:25.397794 16581 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250111 09:01:25.401894 16721 tablet_service.cc:1467] Processing CreateTablet for tablet 53f1dac98fae420e87cef4f3e144f829 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f667fb5fb06d4b0fb1cc730955608895]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:01:25.401921 16996 tablet_service.cc:1467] Processing CreateTablet for tablet 53f1dac98fae420e87cef4f3e144f829 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f667fb5fb06d4b0fb1cc730955608895]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:01:25.402469 16721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53f1dac98fae420e87cef4f3e144f829. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:25.402477 16996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53f1dac98fae420e87cef4f3e144f829. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:25.402380 16859 tablet_service.cc:1467] Processing CreateTablet for tablet 53f1dac98fae420e87cef4f3e144f829 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f667fb5fb06d4b0fb1cc730955608895]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:01:25.402798 16859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53f1dac98fae420e87cef4f3e144f829. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:25.404609 17083 tablet_bootstrap.cc:492] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb: Bootstrap starting.
I20250111 09:01:25.405083 17087 tablet_bootstrap.cc:492] T 53f1dac98fae420e87cef4f3e144f829 P e7bb11a869a542929782a67bbc8a72d8: Bootstrap starting.
I20250111 09:01:25.405596 17083 tablet_bootstrap.cc:654] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:25.406033 17087 tablet_bootstrap.cc:654] T 53f1dac98fae420e87cef4f3e144f829 P e7bb11a869a542929782a67bbc8a72d8: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:25.406191 17084 tablet_bootstrap.cc:492] T 53f1dac98fae420e87cef4f3e144f829 P d5cf50b8a8b940629e21ab677ea44e45: Bootstrap starting.
I20250111 09:01:25.407068 17084 tablet_bootstrap.cc:654] T 53f1dac98fae420e87cef4f3e144f829 P d5cf50b8a8b940629e21ab677ea44e45: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:25.407078 17087 tablet_bootstrap.cc:492] T 53f1dac98fae420e87cef4f3e144f829 P e7bb11a869a542929782a67bbc8a72d8: No bootstrap required, opened a new log
I20250111 09:01:25.407161 17087 ts_tablet_manager.cc:1397] T 53f1dac98fae420e87cef4f3e144f829 P e7bb11a869a542929782a67bbc8a72d8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:01:25.407582 17087 raft_consensus.cc:357] T 53f1dac98fae420e87cef4f3e144f829 P e7bb11a869a542929782a67bbc8a72d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } }
I20250111 09:01:25.407680 17087 raft_consensus.cc:383] T 53f1dac98fae420e87cef4f3e144f829 P e7bb11a869a542929782a67bbc8a72d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:25.407713 17087 raft_consensus.cc:738] T 53f1dac98fae420e87cef4f3e144f829 P e7bb11a869a542929782a67bbc8a72d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7bb11a869a542929782a67bbc8a72d8, State: Initialized, Role: FOLLOWER
I20250111 09:01:25.407835 17087 consensus_queue.cc:260] T 53f1dac98fae420e87cef4f3e144f829 P e7bb11a869a542929782a67bbc8a72d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } }
I20250111 09:01:25.408003 17084 tablet_bootstrap.cc:492] T 53f1dac98fae420e87cef4f3e144f829 P d5cf50b8a8b940629e21ab677ea44e45: No bootstrap required, opened a new log
I20250111 09:01:25.408069 17084 ts_tablet_manager.cc:1397] T 53f1dac98fae420e87cef4f3e144f829 P d5cf50b8a8b940629e21ab677ea44e45: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:01:25.408087 17087 ts_tablet_manager.cc:1428] T 53f1dac98fae420e87cef4f3e144f829 P e7bb11a869a542929782a67bbc8a72d8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250111 09:01:25.408074 17093 tablet_replica.cc:440] T 53f1dac98fae420e87cef4f3e144f829 P e7bb11a869a542929782a67bbc8a72d8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } } }
I20250111 09:01:25.408278 17093 tablet_replica.cc:443] T 53f1dac98fae420e87cef4f3e144f829 P e7bb11a869a542929782a67bbc8a72d8: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:25.408305 17083 tablet_bootstrap.cc:492] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb: No bootstrap required, opened a new log
I20250111 09:01:25.408366 17083 ts_tablet_manager.cc:1397] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20250111 09:01:25.408433 17084 raft_consensus.cc:357] T 53f1dac98fae420e87cef4f3e144f829 P d5cf50b8a8b940629e21ab677ea44e45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } }
I20250111 09:01:25.408505 17084 raft_consensus.cc:383] T 53f1dac98fae420e87cef4f3e144f829 P d5cf50b8a8b940629e21ab677ea44e45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:25.408540 17084 raft_consensus.cc:738] T 53f1dac98fae420e87cef4f3e144f829 P d5cf50b8a8b940629e21ab677ea44e45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5cf50b8a8b940629e21ab677ea44e45, State: Initialized, Role: FOLLOWER
I20250111 09:01:25.408640 17084 consensus_queue.cc:260] T 53f1dac98fae420e87cef4f3e144f829 P d5cf50b8a8b940629e21ab677ea44e45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } }
I20250111 09:01:25.408772 17083 raft_consensus.cc:357] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } }
I20250111 09:01:25.408833 17084 ts_tablet_manager.cc:1428] T 53f1dac98fae420e87cef4f3e144f829 P d5cf50b8a8b940629e21ab677ea44e45: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250111 09:01:25.408849 17083 raft_consensus.cc:383] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:25.408887 17083 raft_consensus.cc:738] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f744ff3c16b04568a8a771904eda03cb, State: Initialized, Role: FOLLOWER
I20250111 09:01:25.408988 17083 consensus_queue.cc:260] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } }
I20250111 09:01:25.409135 17088 tablet_replica.cc:440] T 53f1dac98fae420e87cef4f3e144f829 P d5cf50b8a8b940629e21ab677ea44e45: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } } }
I20250111 09:01:25.409237 17088 tablet_replica.cc:443] T 53f1dac98fae420e87cef4f3e144f829 P d5cf50b8a8b940629e21ab677ea44e45: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:25.409233 17083 ts_tablet_manager.cc:1428] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:01:25.409201 17089 tablet_replica.cc:440] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } } }
I20250111 09:01:25.409307 17089 tablet_replica.cc:443] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:25.423172 17089 raft_consensus.cc:491] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:01:25.423251 17089 raft_consensus.cc:513] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } }
I20250111 09:01:25.423553 17089 leader_election.cc:290] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7bb11a869a542929782a67bbc8a72d8 (127.14.61.1:38471), d5cf50b8a8b940629e21ab677ea44e45 (127.14.61.2:44661)
I20250111 09:01:25.423853 16742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53f1dac98fae420e87cef4f3e144f829" candidate_uuid: "f744ff3c16b04568a8a771904eda03cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7bb11a869a542929782a67bbc8a72d8" is_pre_election: true
I20250111 09:01:25.423907 16879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53f1dac98fae420e87cef4f3e144f829" candidate_uuid: "f744ff3c16b04568a8a771904eda03cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5cf50b8a8b940629e21ab677ea44e45" is_pre_election: true
I20250111 09:01:25.423974 16742 raft_consensus.cc:2463] T 53f1dac98fae420e87cef4f3e144f829 P e7bb11a869a542929782a67bbc8a72d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f744ff3c16b04568a8a771904eda03cb in term 0.
I20250111 09:01:25.424016 16879 raft_consensus.cc:2463] T 53f1dac98fae420e87cef4f3e144f829 P d5cf50b8a8b940629e21ab677ea44e45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f744ff3c16b04568a8a771904eda03cb in term 0.
I20250111 09:01:25.424260 16950 leader_election.cc:304] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7bb11a869a542929782a67bbc8a72d8, f744ff3c16b04568a8a771904eda03cb; no voters:
I20250111 09:01:25.424407 17089 raft_consensus.cc:2798] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:01:25.424455 17089 raft_consensus.cc:491] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:01:25.424489 17089 raft_consensus.cc:3054] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:25.425160 17089 raft_consensus.cc:513] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } }
I20250111 09:01:25.425400 17089 leader_election.cc:290] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb [CANDIDATE]: Term 1 election: Requested vote from peers e7bb11a869a542929782a67bbc8a72d8 (127.14.61.1:38471), d5cf50b8a8b940629e21ab677ea44e45 (127.14.61.2:44661)
I20250111 09:01:25.425684 16742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53f1dac98fae420e87cef4f3e144f829" candidate_uuid: "f744ff3c16b04568a8a771904eda03cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7bb11a869a542929782a67bbc8a72d8"
I20250111 09:01:25.425784 16742 raft_consensus.cc:3054] T 53f1dac98fae420e87cef4f3e144f829 P e7bb11a869a542929782a67bbc8a72d8 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:25.425750 16879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53f1dac98fae420e87cef4f3e144f829" candidate_uuid: "f744ff3c16b04568a8a771904eda03cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5cf50b8a8b940629e21ab677ea44e45"
I20250111 09:01:25.425886 16879 raft_consensus.cc:3054] T 53f1dac98fae420e87cef4f3e144f829 P d5cf50b8a8b940629e21ab677ea44e45 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:25.426390 16742 raft_consensus.cc:2463] T 53f1dac98fae420e87cef4f3e144f829 P e7bb11a869a542929782a67bbc8a72d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f744ff3c16b04568a8a771904eda03cb in term 1.
I20250111 09:01:25.426455 16879 raft_consensus.cc:2463] T 53f1dac98fae420e87cef4f3e144f829 P d5cf50b8a8b940629e21ab677ea44e45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f744ff3c16b04568a8a771904eda03cb in term 1.
I20250111 09:01:25.426640 16950 leader_election.cc:304] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7bb11a869a542929782a67bbc8a72d8, f744ff3c16b04568a8a771904eda03cb; no voters:
I20250111 09:01:25.426789 17089 raft_consensus.cc:2798] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:25.426856 17089 raft_consensus.cc:695] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb [term 1 LEADER]: Becoming Leader. State: Replica: f744ff3c16b04568a8a771904eda03cb, State: Running, Role: LEADER
I20250111 09:01:25.426960 17089 consensus_queue.cc:237] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } }
I20250111 09:01:25.427428 17096 tablet_replica.cc:440] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb: TxnStatusTablet state changed. Reason: New leader f744ff3c16b04568a8a771904eda03cb. Latest consensus state: current_term: 1 leader_uuid: "f744ff3c16b04568a8a771904eda03cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } } }
I20250111 09:01:25.427512 17096 tablet_replica.cc:443] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:25.427695 17110 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:01:25.427865 17110 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:01:25.428124 16582 catalog_manager.cc:5526] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb reported cstate change: term changed from 0 to 1, leader changed from <none> to f744ff3c16b04568a8a771904eda03cb (127.14.61.3). New cstate: current_term: 1 leader_uuid: "f744ff3c16b04568a8a771904eda03cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:25.446478 17096 consensus_queue.cc:1035] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:25.450932 17096 consensus_queue.cc:1035] T e5ba02abf0ab41bb8dd3ec6a4e28422c P f744ff3c16b04568a8a771904eda03cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:25.519356 16742 raft_consensus.cc:1270] T 53f1dac98fae420e87cef4f3e144f829 P e7bb11a869a542929782a67bbc8a72d8 [term 1 FOLLOWER]: Refusing update from remote peer f744ff3c16b04568a8a771904eda03cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:25.519404 16879 raft_consensus.cc:1270] T 53f1dac98fae420e87cef4f3e144f829 P d5cf50b8a8b940629e21ab677ea44e45 [term 1 FOLLOWER]: Refusing update from remote peer f744ff3c16b04568a8a771904eda03cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:25.519749 17096 consensus_queue.cc:1035] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:25.519868 17114 consensus_queue.cc:1035] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:25.522601 17088 tablet_replica.cc:440] T 53f1dac98fae420e87cef4f3e144f829 P d5cf50b8a8b940629e21ab677ea44e45: TxnStatusTablet state changed. Reason: New leader f744ff3c16b04568a8a771904eda03cb. Latest consensus state: current_term: 1 leader_uuid: "f744ff3c16b04568a8a771904eda03cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } } }
I20250111 09:01:25.522692 17088 tablet_replica.cc:443] T 53f1dac98fae420e87cef4f3e144f829 P d5cf50b8a8b940629e21ab677ea44e45: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:25.523218 17093 tablet_replica.cc:440] T 53f1dac98fae420e87cef4f3e144f829 P e7bb11a869a542929782a67bbc8a72d8: TxnStatusTablet state changed. Reason: New leader f744ff3c16b04568a8a771904eda03cb. Latest consensus state: current_term: 1 leader_uuid: "f744ff3c16b04568a8a771904eda03cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } } }
I20250111 09:01:25.523327 17093 tablet_replica.cc:443] T 53f1dac98fae420e87cef4f3e144f829 P e7bb11a869a542929782a67bbc8a72d8: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:25.524488 17096 tablet_replica.cc:440] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f744ff3c16b04568a8a771904eda03cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } } }
I20250111 09:01:25.524567 17096 tablet_replica.cc:443] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:25.524679 17113 mvcc.cc:204] Tried to move back new op lower bound from 7113056606283403264 to 7113056605909839872. Current Snapshot: MvccSnapshot[applied={T|T < 7113056606283403264}]
I20250111 09:01:25.526234 17088 tablet_replica.cc:440] T 53f1dac98fae420e87cef4f3e144f829 P d5cf50b8a8b940629e21ab677ea44e45: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f744ff3c16b04568a8a771904eda03cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } } }
I20250111 09:01:25.526329 17088 tablet_replica.cc:443] T 53f1dac98fae420e87cef4f3e144f829 P d5cf50b8a8b940629e21ab677ea44e45: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:25.528904 17089 tablet_replica.cc:440] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f744ff3c16b04568a8a771904eda03cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } } }
I20250111 09:01:25.528995 17089 tablet_replica.cc:443] T 53f1dac98fae420e87cef4f3e144f829 P f744ff3c16b04568a8a771904eda03cb: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:25.529471 17117 mvcc.cc:204] Tried to move back new op lower bound from 7113056606283403264 to 7113056605909839872. Current Snapshot: MvccSnapshot[applied={T|T < 7113056606283403264}]
I20250111 09:01:25.529709 17093 tablet_replica.cc:440] T 53f1dac98fae420e87cef4f3e144f829 P e7bb11a869a542929782a67bbc8a72d8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f744ff3c16b04568a8a771904eda03cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 } } peers { permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 } } peers { permanent_uuid: "f744ff3c16b04568a8a771904eda03cb" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 35667 } } }
I20250111 09:01:25.529821 17093 tablet_replica.cc:443] T 53f1dac98fae420e87cef4f3e144f829 P e7bb11a869a542929782a67bbc8a72d8: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:25.560230 16879 raft_consensus.cc:1270] T 358e989064ec4d7b8708607b6a0da029 P d5cf50b8a8b940629e21ab677ea44e45 [term 1 FOLLOWER]: Refusing update from remote peer f744ff3c16b04568a8a771904eda03cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:25.560230 16742 raft_consensus.cc:1270] T 358e989064ec4d7b8708607b6a0da029 P e7bb11a869a542929782a67bbc8a72d8 [term 1 FOLLOWER]: Refusing update from remote peer f744ff3c16b04568a8a771904eda03cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:25.560578 17129 consensus_queue.cc:1035] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5cf50b8a8b940629e21ab677ea44e45" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:25.560703 17114 consensus_queue.cc:1035] T 358e989064ec4d7b8708607b6a0da029 P f744ff3c16b04568a8a771904eda03cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7bb11a869a542929782a67bbc8a72d8" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:25.564121 17115 mvcc.cc:204] Tried to move back new op lower bound from 7113056606452047872 to 7113056604485128192. Current Snapshot: MvccSnapshot[applied={T|T < 7113056606452047872}]
I20250111 09:01:28.630204 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 16658
I20250111 09:01:28.635985 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 16795
I20250111 09:01:28.642302 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 16932
I20250111 09:01:28.647542 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 16552
2025-01-11T09:01:28Z chronyd exiting
[ OK ] TxnWriteOpsITest.TxnMultipleSingleRowWritesCommit (4493 ms)
[ RUN ] TxnWriteOpsITest.FrequentElections
2025-01-11T09:01:28Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-01-11T09:01:28Z Disabled control of system clock
I20250111 09:01:28.675451 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.61.62:35629
--webserver_interface=127.14.61.62
--webserver_port=0
--builtin_ntp_servers=127.14.61.20:34855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.61.62:35629
--txn_manager_enabled=true
--txn_manager_lazily_initialized=false with env {}
W20250111 09:01:28.763823 17150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:01:28.763988 17150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:01:28.764031 17150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:01:28.766888 17150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250111 09:01:28.766928 17150 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20250111 09:01:28.766943 17150 flags.cc:425] Enabled experimental flag: --txn_manager_lazily_initialized=false
W20250111 09:01:28.766957 17150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:01:28.766973 17150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250111 09:01:28.766988 17150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250111 09:01:28.770016 17150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:34855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.61.62:35629
--txn_manager_enabled=true
--txn_manager_lazily_initialized=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.61.62:35629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.61.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.17150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:01:28.770766 17150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:01:28.771612 17150 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:01:28.777199 17156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:28.777320 17155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:28.777976 17158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:01:28.778599 17150 server_base.cc:1034] running on GCE node
I20250111 09:01:28.779000 17150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:01:28.779693 17150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:01:28.780851 17150 hybrid_clock.cc:648] HybridClock initialized: now 1736586088780832 us; error 31 us; skew 500 ppm
I20250111 09:01:28.782291 17150 webserver.cc:458] Webserver started at http://127.14.61.62:36095/ using document root <none> and password file <none>
I20250111 09:01:28.782763 17150 fs_manager.cc:362] Metadata directory not provided
I20250111 09:01:28.782816 17150 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:01:28.782965 17150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:01:28.784365 17150 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/master-0/data/instance:
uuid: "ad245100aabd40d788896e9d17828393"
format_stamp: "Formatted at 2025-01-11 09:01:28 on dist-test-slave-nvbt"
I20250111 09:01:28.784821 17150 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/master-0/wal/instance:
uuid: "ad245100aabd40d788896e9d17828393"
format_stamp: "Formatted at 2025-01-11 09:01:28 on dist-test-slave-nvbt"
I20250111 09:01:28.787632 17150 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20250111 09:01:28.789289 17164 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:28.790040 17150 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20250111 09:01:28.790133 17150 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/master-0/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/master-0/wal
uuid: "ad245100aabd40d788896e9d17828393"
format_stamp: "Formatted at 2025-01-11 09:01:28 on dist-test-slave-nvbt"
I20250111 09:01:28.790195 17150 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:01:28.821259 17150 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:01:28.821673 17150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:01:28.821805 17150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:01:28.828078 17150 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:35629
I20250111 09:01:28.828087 17226 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:35629 every 8 connection(s)
I20250111 09:01:28.828779 17150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
I20250111 09:01:28.829356 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 17150
I20250111 09:01:28.829461 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/master-0/wal/instance
I20250111 09:01:28.831125 17227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:28.835816 17227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393: Bootstrap starting.
I20250111 09:01:28.837635 17227 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:28.838152 17227 log.cc:826] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393: Log is configured to *not* fsync() on all Append() calls
I20250111 09:01:28.839349 17227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393: No bootstrap required, opened a new log
I20250111 09:01:28.841686 17227 raft_consensus.cc:357] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad245100aabd40d788896e9d17828393" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 35629 } }
I20250111 09:01:28.841823 17227 raft_consensus.cc:383] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:28.841863 17227 raft_consensus.cc:738] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad245100aabd40d788896e9d17828393, State: Initialized, Role: FOLLOWER
I20250111 09:01:28.842157 17227 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad245100aabd40d788896e9d17828393" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 35629 } }
I20250111 09:01:28.842249 17227 raft_consensus.cc:397] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:01:28.842285 17227 raft_consensus.cc:491] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:01:28.842350 17227 raft_consensus.cc:3054] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:28.842926 17227 raft_consensus.cc:513] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad245100aabd40d788896e9d17828393" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 35629 } }
I20250111 09:01:28.843170 17227 leader_election.cc:304] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad245100aabd40d788896e9d17828393; no voters:
I20250111 09:01:28.843387 17227 leader_election.cc:290] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:01:28.843489 17232 raft_consensus.cc:2798] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:28.843667 17232 raft_consensus.cc:695] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393 [term 1 LEADER]: Becoming Leader. State: Replica: ad245100aabd40d788896e9d17828393, State: Running, Role: LEADER
I20250111 09:01:28.843940 17232 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad245100aabd40d788896e9d17828393" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 35629 } }
I20250111 09:01:28.844295 17227 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393 [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:01:28.845268 17234 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad245100aabd40d788896e9d17828393. Latest consensus state: current_term: 1 leader_uuid: "ad245100aabd40d788896e9d17828393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad245100aabd40d788896e9d17828393" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 35629 } } }
I20250111 09:01:28.845381 17234 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393 [sys.catalog]: This master's current role is: LEADER
I20250111 09:01:28.845645 17233 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad245100aabd40d788896e9d17828393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad245100aabd40d788896e9d17828393" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 35629 } } }
I20250111 09:01:28.845753 17233 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393 [sys.catalog]: This master's current role is: LEADER
I20250111 09:01:28.845781 17241 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:01:28.848024 17241 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:01:28.852272 17241 catalog_manager.cc:1348] Generated new cluster ID: b143574d0ea947e498881c8304012086
I20250111 09:01:28.852316 17241 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:01:28.870446 17241 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:01:28.871100 17241 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:01:28.880515 17241 catalog_manager.cc:5899] T 00000000000000000000000000000000 P ad245100aabd40d788896e9d17828393: Generated new TSK 0
I20250111 09:01:28.880970 17241 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:01:28.889678 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.1:0
--local_ip_for_outbound_sockets=127.14.61.1
--webserver_interface=127.14.61.1
--webserver_port=0
--tserver_master_addrs=127.14.61.62:35629
--builtin_ntp_servers=127.14.61.20:34855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_enable_pre_election=false
--heartbeat_interval_ms=16
--enable_txn_partition_lock=false
--enable_txn_system_client_init=true with env {}
I20250111 09:01:28.889072 17180 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54446:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20250111 09:01:28.893174 17227 master.cc:440] Invalid argument: unable to initialize TxnManager: Error creating table kudu_system.kudu_transactions on the master: not enough live tablet servers to create a table with the requested replication factor 3; 0 tablet servers are alive: unable to init TxnManager, will retry in 1.000s
W20250111 09:01:28.989604 17256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:01:28.989771 17256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:01:28.989823 17256 flags.cc:425] Enabled unsafe flag: --enable_txn_partition_lock=false
W20250111 09:01:28.989871 17256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:01:28.992698 17256 flags.cc:425] Enabled experimental flag: --raft_enable_pre_election=false
W20250111 09:01:28.992760 17256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:01:28.992851 17256 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:01:28.992923 17256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.1
I20250111 09:01:28.996351 17256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:34855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_enable_pre_election=false
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.61.1
--webserver_port=0
--enable_txn_partition_lock=false
--enable_txn_system_client_init=true
--heartbeat_interval_ms=16
--tserver_master_addrs=127.14.61.62:35629
--never_fsync=true
--heap_profile_path=/tmp/kudu.17256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.1
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:01:28.997202 17256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:01:28.998114 17256 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:01:29.004967 17262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:29.005596 17264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:29.005848 17261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:01:29.006546 17256 server_base.cc:1034] running on GCE node
I20250111 09:01:29.006906 17256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:01:29.007376 17256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:01:29.008569 17256 hybrid_clock.cc:648] HybridClock initialized: now 1736586089008558 us; error 23 us; skew 500 ppm
I20250111 09:01:29.010115 17256 webserver.cc:458] Webserver started at http://127.14.61.1:44113/ using document root <none> and password file <none>
I20250111 09:01:29.010627 17256 fs_manager.cc:362] Metadata directory not provided
I20250111 09:01:29.010685 17256 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:01:29.010852 17256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:01:29.012292 17256 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-0/data/instance:
uuid: "fb270e42d7c9400f8c05bb9565db1972"
format_stamp: "Formatted at 2025-01-11 09:01:29 on dist-test-slave-nvbt"
I20250111 09:01:29.012681 17256 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-0/wal/instance:
uuid: "fb270e42d7c9400f8c05bb9565db1972"
format_stamp: "Formatted at 2025-01-11 09:01:29 on dist-test-slave-nvbt"
I20250111 09:01:29.015659 17256 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20250111 09:01:29.017484 17270 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:29.018335 17256 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:01:29.018428 17256 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-0/wal
uuid: "fb270e42d7c9400f8c05bb9565db1972"
format_stamp: "Formatted at 2025-01-11 09:01:29 on dist-test-slave-nvbt"
I20250111 09:01:29.018520 17256 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:01:29.032295 17256 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:01:29.032754 17256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:01:29.032895 17256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:01:29.034185 17256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:01:29.034248 17256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:29.034296 17256 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:01:29.034322 17256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:29.043946 17256 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:43619
I20250111 09:01:29.044006 17389 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:43619 every 8 connection(s)
I20250111 09:01:29.044955 17256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
I20250111 09:01:29.046272 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 17256
I20250111 09:01:29.046290 17391 heartbeater.cc:346] Connected to a master server at 127.14.61.62:35629
I20250111 09:01:29.046386 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-0/wal/instance
I20250111 09:01:29.046408 17391 heartbeater.cc:463] Registering TS with master...
I20250111 09:01:29.046947 17391 heartbeater.cc:510] Master 127.14.61.62:35629 requested a full tablet report, sending...
I20250111 09:01:29.048229 17181 ts_manager.cc:194] Registered new tserver with Master: fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619)
I20250111 09:01:29.048727 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.2:0
--local_ip_for_outbound_sockets=127.14.61.2
--webserver_interface=127.14.61.2
--webserver_port=0
--tserver_master_addrs=127.14.61.62:35629
--builtin_ntp_servers=127.14.61.20:34855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_enable_pre_election=false
--heartbeat_interval_ms=16
--enable_txn_partition_lock=false
--enable_txn_system_client_init=true with env {}
I20250111 09:01:29.049503 17181 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.1:38649
I20250111 09:01:29.067399 17391 heartbeater.cc:502] Master 127.14.61.62:35629 was elected leader, sending a full tablet report...
W20250111 09:01:29.143787 17393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:01:29.143956 17393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:01:29.143990 17393 flags.cc:425] Enabled unsafe flag: --enable_txn_partition_lock=false
W20250111 09:01:29.144029 17393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:01:29.147066 17393 flags.cc:425] Enabled experimental flag: --raft_enable_pre_election=false
W20250111 09:01:29.147122 17393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:01:29.147166 17393 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:01:29.147212 17393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.2
I20250111 09:01:29.150465 17393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:34855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_enable_pre_election=false
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.61.2
--webserver_port=0
--enable_txn_partition_lock=false
--enable_txn_system_client_init=true
--heartbeat_interval_ms=16
--tserver_master_addrs=127.14.61.62:35629
--never_fsync=true
--heap_profile_path=/tmp/kudu.17393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.2
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:01:29.151234 17393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:01:29.152141 17393 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:01:29.159214 17398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:01:29.159946 17393 server_base.cc:1034] running on GCE node
W20250111 09:01:29.160188 17401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:29.160728 17399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:01:29.161078 17393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:01:29.161468 17393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:01:29.162592 17393 hybrid_clock.cc:648] HybridClock initialized: now 1736586089162573 us; error 29 us; skew 500 ppm
I20250111 09:01:29.164160 17393 webserver.cc:458] Webserver started at http://127.14.61.2:34151/ using document root <none> and password file <none>
I20250111 09:01:29.164642 17393 fs_manager.cc:362] Metadata directory not provided
I20250111 09:01:29.164692 17393 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:01:29.164856 17393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:01:29.166249 17393 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-1/data/instance:
uuid: "262623ac6df745c785ce267ea39bd265"
format_stamp: "Formatted at 2025-01-11 09:01:29 on dist-test-slave-nvbt"
I20250111 09:01:29.166647 17393 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-1/wal/instance:
uuid: "262623ac6df745c785ce267ea39bd265"
format_stamp: "Formatted at 2025-01-11 09:01:29 on dist-test-slave-nvbt"
I20250111 09:01:29.169533 17393 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20250111 09:01:29.171509 17407 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:29.172277 17393 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:01:29.172353 17393 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-1/wal
uuid: "262623ac6df745c785ce267ea39bd265"
format_stamp: "Formatted at 2025-01-11 09:01:29 on dist-test-slave-nvbt"
I20250111 09:01:29.172425 17393 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:01:29.183617 17393 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:01:29.184072 17393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:01:29.184207 17393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:01:29.185438 17393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:01:29.185488 17393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:29.185539 17393 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:01:29.185575 17393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:29.195859 17393 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.2:38607
I20250111 09:01:29.195909 17527 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.2:38607 every 8 connection(s)
I20250111 09:01:29.196686 17393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
I20250111 09:01:29.197818 17528 heartbeater.cc:346] Connected to a master server at 127.14.61.62:35629
I20250111 09:01:29.197932 17528 heartbeater.cc:463] Registering TS with master...
I20250111 09:01:29.198251 17528 heartbeater.cc:510] Master 127.14.61.62:35629 requested a full tablet report, sending...
I20250111 09:01:29.199083 17181 ts_manager.cc:194] Registered new tserver with Master: 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.199517 17181 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.2:33927
I20250111 09:01:29.204368 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 17393
I20250111 09:01:29.204485 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-1/wal/instance
I20250111 09:01:29.206162 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.3:0
--local_ip_for_outbound_sockets=127.14.61.3
--webserver_interface=127.14.61.3
--webserver_port=0
--tserver_master_addrs=127.14.61.62:35629
--builtin_ntp_servers=127.14.61.20:34855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_enable_pre_election=false
--heartbeat_interval_ms=16
--enable_txn_partition_lock=false
--enable_txn_system_client_init=true with env {}
I20250111 09:01:29.216881 17528 heartbeater.cc:502] Master 127.14.61.62:35629 was elected leader, sending a full tablet report...
W20250111 09:01:29.304723 17530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:01:29.304914 17530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:01:29.304975 17530 flags.cc:425] Enabled unsafe flag: --enable_txn_partition_lock=false
W20250111 09:01:29.305047 17530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:01:29.308137 17530 flags.cc:425] Enabled experimental flag: --raft_enable_pre_election=false
W20250111 09:01:29.308199 17530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:01:29.308261 17530 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:01:29.308328 17530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.3
I20250111 09:01:29.311642 17530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:34855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_enable_pre_election=false
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.61.3
--webserver_port=0
--enable_txn_partition_lock=false
--enable_txn_system_client_init=true
--heartbeat_interval_ms=16
--tserver_master_addrs=127.14.61.62:35629
--never_fsync=true
--heap_profile_path=/tmp/kudu.17530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.3
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:01:29.312564 17530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:01:29.313548 17530 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:01:29.320326 17536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:29.320567 17535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:01:29.321239 17538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:01:29.322070 17530 server_base.cc:1034] running on GCE node
I20250111 09:01:29.322396 17530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:01:29.322957 17530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:01:29.324108 17530 hybrid_clock.cc:648] HybridClock initialized: now 1736586089324093 us; error 25 us; skew 500 ppm
I20250111 09:01:29.325821 17530 webserver.cc:458] Webserver started at http://127.14.61.3:37133/ using document root <none> and password file <none>
I20250111 09:01:29.326308 17530 fs_manager.cc:362] Metadata directory not provided
I20250111 09:01:29.326362 17530 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:01:29.326548 17530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:01:29.328008 17530 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-2/data/instance:
uuid: "b5fb421fa3354606aecbeafe55a0565c"
format_stamp: "Formatted at 2025-01-11 09:01:29 on dist-test-slave-nvbt"
I20250111 09:01:29.328413 17530 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-2/wal/instance:
uuid: "b5fb421fa3354606aecbeafe55a0565c"
format_stamp: "Formatted at 2025-01-11 09:01:29 on dist-test-slave-nvbt"
I20250111 09:01:29.331279 17530 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20250111 09:01:29.333020 17544 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:29.333886 17530 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:01:29.333990 17530 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-2/wal
uuid: "b5fb421fa3354606aecbeafe55a0565c"
format_stamp: "Formatted at 2025-01-11 09:01:29 on dist-test-slave-nvbt"
I20250111 09:01:29.334064 17530 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:01:29.350015 17530 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:01:29.350425 17530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:01:29.350564 17530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:01:29.351722 17530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:01:29.351775 17530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:29.351831 17530 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:01:29.351857 17530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:01:29.361372 17530 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.3:43257
I20250111 09:01:29.361441 17663 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.3:43257 every 8 connection(s)
I20250111 09:01:29.362131 17530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
I20250111 09:01:29.363128 17665 heartbeater.cc:346] Connected to a master server at 127.14.61.62:35629
I20250111 09:01:29.363233 17665 heartbeater.cc:463] Registering TS with master...
I20250111 09:01:29.363637 17665 heartbeater.cc:510] Master 127.14.61.62:35629 requested a full tablet report, sending...
I20250111 09:01:29.364348 17181 ts_manager.cc:194] Registered new tserver with Master: b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.364828 17181 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.3:49161
I20250111 09:01:29.370765 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 17530
I20250111 09:01:29.370880 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.FrequentElections.1736586070622223-14580-0/minicluster-data/ts-2/wal/instance
I20250111 09:01:29.372579 14580 external_mini_cluster.cc:934] 3 TS(s) registered with all masters
I20250111 09:01:29.382576 17665 heartbeater.cc:502] Master 127.14.61.62:35629 was elected leader, sending a full tablet report...
I20250111 09:01:29.382874 17180 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54462:
name: "txn_write_ops_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
W20250111 09:01:29.383564 17180 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table txn_write_ops_test in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250111 09:01:29.397811 17319 tablet_service.cc:1467] Processing CreateTablet for tablet 6ab63c6543ad4a0ab83e76267a725edd (DEFAULT_TABLE table=txn_write_ops_test [id=2af520f966b34e62b18c86445c0c4d49]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:29.397889 17457 tablet_service.cc:1467] Processing CreateTablet for tablet 2a4e17d0e2894eaa95b650f1af0ce650 (DEFAULT_TABLE table=txn_write_ops_test [id=2af520f966b34e62b18c86445c0c4d49]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:29.397815 17320 tablet_service.cc:1467] Processing CreateTablet for tablet 2a4e17d0e2894eaa95b650f1af0ce650 (DEFAULT_TABLE table=txn_write_ops_test [id=2af520f966b34e62b18c86445c0c4d49]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:29.398653 17457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a4e17d0e2894eaa95b650f1af0ce650. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:29.398653 17320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a4e17d0e2894eaa95b650f1af0ce650. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:29.398947 17319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ab63c6543ad4a0ab83e76267a725edd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:29.402220 17594 tablet_service.cc:1467] Processing CreateTablet for tablet 2a4e17d0e2894eaa95b650f1af0ce650 (DEFAULT_TABLE table=txn_write_ops_test [id=2af520f966b34e62b18c86445c0c4d49]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:29.402877 17594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a4e17d0e2894eaa95b650f1af0ce650. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:29.404210 17682 tablet_bootstrap.cc:492] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972: Bootstrap starting.
I20250111 09:01:29.405472 17593 tablet_service.cc:1467] Processing CreateTablet for tablet 6ab63c6543ad4a0ab83e76267a725edd (DEFAULT_TABLE table=txn_write_ops_test [id=2af520f966b34e62b18c86445c0c4d49]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:29.405736 17593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ab63c6543ad4a0ab83e76267a725edd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:29.406770 17682 tablet_bootstrap.cc:654] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:29.407610 17682 log.cc:826] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972: Log is configured to *not* fsync() on all Append() calls
I20250111 09:01:29.407763 17455 tablet_service.cc:1467] Processing CreateTablet for tablet 6ab63c6543ad4a0ab83e76267a725edd (DEFAULT_TABLE table=txn_write_ops_test [id=2af520f966b34e62b18c86445c0c4d49]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:01:29.407994 17455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ab63c6543ad4a0ab83e76267a725edd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:29.408071 17683 tablet_bootstrap.cc:492] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Bootstrap starting.
I20250111 09:01:29.409405 17682 tablet_bootstrap.cc:492] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972: No bootstrap required, opened a new log
I20250111 09:01:29.409566 17682 ts_tablet_manager.cc:1397] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20250111 09:01:29.409969 17683 tablet_bootstrap.cc:654] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:29.410526 17683 log.cc:826] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Log is configured to *not* fsync() on all Append() calls
I20250111 09:01:29.411208 17686 tablet_bootstrap.cc:492] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Bootstrap starting.
I20250111 09:01:29.411937 17683 tablet_bootstrap.cc:492] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: No bootstrap required, opened a new log
I20250111 09:01:29.412081 17683 ts_tablet_manager.cc:1397] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20250111 09:01:29.412611 17682 raft_consensus.cc:357] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.412810 17682 raft_consensus.cc:383] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:29.412883 17682 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Initialized, Role: FOLLOWER
I20250111 09:01:29.413272 17682 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.414124 17686 tablet_bootstrap.cc:654] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:29.414163 17682 ts_tablet_manager.cc:1428] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20250111 09:01:29.414472 17682 tablet_bootstrap.cc:492] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Bootstrap starting.
I20250111 09:01:29.414839 17686 log.cc:826] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Log is configured to *not* fsync() on all Append() calls
I20250111 09:01:29.415498 17682 tablet_bootstrap.cc:654] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:29.416319 17686 tablet_bootstrap.cc:492] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: No bootstrap required, opened a new log
I20250111 09:01:29.416344 17682 tablet_bootstrap.cc:492] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: No bootstrap required, opened a new log
I20250111 09:01:29.416402 17682 ts_tablet_manager.cc:1397] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:01:29.416512 17686 ts_tablet_manager.cc:1397] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20250111 09:01:29.416759 17682 raft_consensus.cc:357] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.416846 17682 raft_consensus.cc:383] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:29.416870 17682 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Initialized, Role: FOLLOWER
I20250111 09:01:29.416958 17682 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.417191 17682 ts_tablet_manager.cc:1428] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:01:29.418095 17683 raft_consensus.cc:357] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.418421 17683 raft_consensus.cc:383] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:29.418489 17683 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Initialized, Role: FOLLOWER
I20250111 09:01:29.418848 17683 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.419255 17686 raft_consensus.cc:357] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.419462 17686 raft_consensus.cc:383] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:29.419543 17686 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Initialized, Role: FOLLOWER
I20250111 09:01:29.419785 17683 ts_tablet_manager.cc:1428] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Time spent starting tablet: real 0.008s user 0.004s sys 0.001s
I20250111 09:01:29.419983 17686 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.420089 17683 tablet_bootstrap.cc:492] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Bootstrap starting.
I20250111 09:01:29.420640 17686 ts_tablet_manager.cc:1428] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20250111 09:01:29.420881 17683 tablet_bootstrap.cc:654] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:29.420964 17686 tablet_bootstrap.cc:492] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Bootstrap starting.
I20250111 09:01:29.421867 17686 tablet_bootstrap.cc:654] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:29.422051 17683 tablet_bootstrap.cc:492] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: No bootstrap required, opened a new log
I20250111 09:01:29.422127 17683 ts_tablet_manager.cc:1397] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:01:29.422530 17683 raft_consensus.cc:357] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.422601 17683 raft_consensus.cc:383] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:29.422621 17683 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Initialized, Role: FOLLOWER
I20250111 09:01:29.422715 17683 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.422917 17683 ts_tablet_manager.cc:1428] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:01:29.423105 17686 tablet_bootstrap.cc:492] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: No bootstrap required, opened a new log
I20250111 09:01:29.423173 17686 ts_tablet_manager.cc:1397] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:01:29.423584 17686 raft_consensus.cc:357] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.423660 17686 raft_consensus.cc:383] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:29.423682 17686 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Initialized, Role: FOLLOWER
I20250111 09:01:29.423799 17686 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.424029 17686 ts_tablet_manager.cc:1428] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20250111 09:01:29.447794 17529 tablet.cc:2367] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250111 09:01:29.545917 17392 tablet.cc:2367] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250111 09:01:29.564417 17691 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:01:29.564553 17691 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:29.565480 17691 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.566438 17691 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 1 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.569355 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.569538 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:29.569592 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.569762 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:29.570607 17340 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 1.
I20250111 09:01:29.570784 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 1.
I20250111 09:01:29.570986 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:29.571173 17691 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:29.571337 17691 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 1 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.571592 17691 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.573992 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 0 to 1, leader changed from <none> to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 1 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:29.612825 17666 tablet.cc:2367] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250111 09:01:29.675693 17687 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:01:29.675776 17687 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:29.676570 17687 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.677616 17687 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 1 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.681252 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.681310 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.681398 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:29.681428 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:29.682181 17467 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 1.
I20250111 09:01:29.682251 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 1.
I20250111 09:01:29.682502 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:29.682691 17687 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:29.682832 17687 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 1 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.683079 17687 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.685336 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.766042 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
W20250111 09:01:29.766837 17669 txn_manager_proxy_rpc.cc:150] re-attempting BeginTransaction request to one of TxnManagers
I20250111 09:01:29.769447 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.769595 17340 raft_consensus.cc:474] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 1 LEADER]: Not starting forced leader election -- already a leader
I20250111 09:01:29.770144 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.770236 17340 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.770294 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 1 FOLLOWER]: Advancing to term 2
I20250111 09:01:29.771042 17340 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.771358 17340 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 2 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.771718 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.771751 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.771847 17467 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 1 LEADER]: Stepping down as leader of term 1
I20250111 09:01:29.771864 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:29.771864 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 1 FOLLOWER]: Advancing to term 2
I20250111 09:01:29.771895 17467 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.772040 17467 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.772279 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 1 FOLLOWER]: Advancing to term 2
I20250111 09:01:29.772786 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 2.
I20250111 09:01:29.773094 17467 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.773190 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:29.773345 17687 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 2 FOLLOWER]: Leader election won for term 2
I20250111 09:01:29.773418 17687 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 2 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.773566 17687 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.774899 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 1 to 2, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 2 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.775596 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.775707 17467 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.775761 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 1 FOLLOWER]: Advancing to term 2
I20250111 09:01:29.776376 17467 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.776680 17467 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 2 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.776948 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.777051 17340 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 1 LEADER]: Stepping down as leader of term 1
I20250111 09:01:29.777101 17340 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.777114 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.777215 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 1 FOLLOWER]: Advancing to term 2
I20250111 09:01:29.777216 17340 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.777438 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 1 FOLLOWER]: Advancing to term 2
I20250111 09:01:29.777442 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.777524 17467 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.777575 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 2 FOLLOWER]: Advancing to term 3
I20250111 09:01:29.777940 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 2.
I20250111 09:01:29.778230 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:29.778334 17340 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.778398 17691 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 2 FOLLOWER]: Leader election won for term 2
I20250111 09:01:29.778458 17691 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 2 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.778564 17691 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.779352 17467 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.779867 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.779956 17340 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 2 LEADER]: Stepping down as leader of term 2
I20250111 09:01:29.779945 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.779992 17340 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.780035 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 2 FOLLOWER]: Advancing to term 3
I20250111 09:01:29.780089 17340 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.780198 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 2 FOLLOWER]: Advancing to term 3
I20250111 09:01:29.780668 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 3.
I20250111 09:01:29.780619 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 1 to 2, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 2 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.780841 17340 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 3 because replica has last-logged OpId of term: 2 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20250111 09:01:29.780886 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:29.781046 17695 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 3 FOLLOWER]: Leader election won for term 3
I20250111 09:01:29.781114 17695 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 3 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.781209 17695 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.780812 17467 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 3 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.782014 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:29.783193 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 2 to 3, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 3 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.785533 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:29.785651 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.785701 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 2 FOLLOWER]: Advancing to term 3
I20250111 09:01:29.786262 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.787060 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 3 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.788029 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:29.788116 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.788156 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 3 FOLLOWER]: Advancing to term 4
I20250111 09:01:29.788771 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.789081 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 4 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.789426 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:29.790133 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.790227 17340 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.790273 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.790397 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.790275 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 2 FOLLOWER]: Advancing to term 3
I20250111 09:01:29.790479 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 3 FOLLOWER]: Advancing to term 4
I20250111 09:01:29.790513 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.790553 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.790592 17467 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 2 LEADER]: Stepping down as leader of term 2
I20250111 09:01:29.790624 17467 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.790643 17476 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 3 LEADER]: Stepping down as leader of term 3
I20250111 09:01:29.790676 17476 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.790715 17467 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.790813 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 2 FOLLOWER]: Advancing to term 3
I20250111 09:01:29.790786 17476 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.790897 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 3 FOLLOWER]: Advancing to term 4
I20250111 09:01:29.791141 17340 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.791368 17338 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 4 because replica has last-logged OpId of term: 2 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.791429 17340 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 3 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.791507 17467 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 3 because replica has last-logged OpId of term: 2 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.791721 17339 raft_consensus.cc:2388] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c in current term 3: Already voted for candidate fb270e42d7c9400f8c05bb9565db1972 in this term.
I20250111 09:01:29.791761 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.792048 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.792078 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.792136 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.792202 17614 raft_consensus.cc:2388] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 in current term 3: Already voted for candidate b5fb421fa3354606aecbeafe55a0565c in this term.
I20250111 09:01:29.792209 17340 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.792250 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 4 FOLLOWER]: Advancing to term 5
I20250111 09:01:29.792407 17690 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20250111 09:01:29.792421 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.792521 17476 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 3 because replica has last-logged OpId of term: 2 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20250111 09:01:29.792505 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.792644 17690 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20250111 09:01:29.792754 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:29.792904 17687 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20250111 09:01:29.792913 17340 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.793229 17340 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 5 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.793632 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:29.793749 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.793751 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.793839 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 4 FOLLOWER]: Advancing to term 5
I20250111 09:01:29.793843 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 4 FOLLOWER]: Advancing to term 5
I20250111 09:01:29.794082 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.794169 17467 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.794214 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 3 FOLLOWER]: Advancing to term 4
I20250111 09:01:29.794541 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 5.
I20250111 09:01:29.794649 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20250111 09:01:29.794788 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:29.794914 17687 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 5 FOLLOWER]: Leader election won for term 5
I20250111 09:01:29.794925 17467 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.794983 17687 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 5 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.795102 17687 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.795189 17467 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 4 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.795488 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.795498 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.795575 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 3 FOLLOWER]: Advancing to term 4
I20250111 09:01:29.795580 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 3 FOLLOWER]: Advancing to term 4
I20250111 09:01:29.795964 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.796051 17467 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.796093 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 5 FOLLOWER]: Advancing to term 6
I20250111 09:01:29.796178 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 4.
I20250111 09:01:29.796808 17467 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.796903 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:29.797072 17691 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 4 FOLLOWER]: Leader election won for term 4
I20250111 09:01:29.797140 17691 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 4 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.797152 17467 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 6 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.797210 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 3 to 5, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 5 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.797240 17691 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.798106 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.797813 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.798231 17339 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 5 LEADER]: Stepping down as leader of term 5
I20250111 09:01:29.798269 17339 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.798370 17339 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.798475 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 5 FOLLOWER]: Advancing to term 6
I20250111 09:01:29.798506 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 5 FOLLOWER]: Advancing to term 6
I20250111 09:01:29.798493 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.799182 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 6.
I20250111 09:01:29.799225 17339 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 6 because replica has last-logged OpId of term: 5 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20250111 09:01:29.799391 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:29.799443 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:29.799518 17695 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 6 FOLLOWER]: Leader election won for term 6
I20250111 09:01:29.799576 17695 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 6 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.799662 17695 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.796253 17340 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 4.
I20250111 09:01:29.799850 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:29.799925 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.799973 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 4 FOLLOWER]: Advancing to term 5
I20250111 09:01:29.800570 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.800985 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 5 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.801285 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 5 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.801366 17467 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 4 LEADER]: Stepping down as leader of term 4
I20250111 09:01:29.801398 17467 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.801388 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 5 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.801467 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 4 FOLLOWER]: Advancing to term 5
I20250111 09:01:29.801506 17467 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.801493 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 5 to 6, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 6 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.801543 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:29.801596 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 4 FOLLOWER]: Advancing to term 5
I20250111 09:01:29.801622 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.801663 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 6 FOLLOWER]: Advancing to term 7
I20250111 09:01:29.802155 17340 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 5 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.802237 17467 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.802266 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.802511 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.802699 17690 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20250111 09:01:29.802784 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 7 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.802860 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 6 FOLLOWER]: Advancing to term 7
I20250111 09:01:29.802903 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 7 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.803174 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:29.803572 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.803620 17339 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 7 because replica has last-logged OpId of term: 5 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.803658 17340 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.803694 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 5 FOLLOWER]: Advancing to term 6
I20250111 09:01:29.803926 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 7 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.804019 17467 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 6 LEADER]: Stepping down as leader of term 6
I20250111 09:01:29.804069 17467 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.804195 17467 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.804308 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 6 FOLLOWER]: Advancing to term 7
I20250111 09:01:29.804378 17340 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.804682 17340 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 6 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.804980 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.805095 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 5 FOLLOWER]: Advancing to term 6
I20250111 09:01:29.805083 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.805172 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 5 FOLLOWER]: Advancing to term 6
I20250111 09:01:29.804983 17467 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.805507 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.805558 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.805594 17340 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.805640 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 7 FOLLOWER]: Advancing to term 8
I20250111 09:01:29.805783 17690 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20250111 09:01:29.805872 17476 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20250111 09:01:29.805933 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 6.
I20250111 09:01:29.806154 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:01:29.806375 17687 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 6 FOLLOWER]: Leader election won for term 6
I20250111 09:01:29.806346 17340 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.806494 17687 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 6 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.806596 17687 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.806694 17340 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 8 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.806968 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.806974 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.807057 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 7 FOLLOWER]: Advancing to term 8
I20250111 09:01:29.807140 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 7 FOLLOWER]: Advancing to term 8
I20250111 09:01:29.807232 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:29.807612 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.807729 17476 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.807785 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 6 FOLLOWER]: Advancing to term 7
I20250111 09:01:29.807799 17467 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 2.
I20250111 09:01:29.808063 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 8.
I20250111 09:01:29.808039 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 4 to 6, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 6 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.808259 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:01:29.808416 17699 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 8 FOLLOWER]: Leader election won for term 8
I20250111 09:01:29.808503 17699 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 8 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.808543 17476 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.808611 17699 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 5.2, Last appended by leader: 2, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.808818 17476 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 7 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.809191 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.809331 17340 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 6 LEADER]: Stepping down as leader of term 6
I20250111 09:01:29.809378 17340 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.809460 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.809497 17340 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 6.2, Last appended by leader: 2, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.809545 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 6 FOLLOWER]: Advancing to term 7
I20250111 09:01:29.809625 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 6 FOLLOWER]: Advancing to term 7
I20250111 09:01:29.810381 17340 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 7 because replica has last-logged OpId of term: 6 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20250111 09:01:29.810559 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 7.
I20250111 09:01:29.809410 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.810809 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.810912 17476 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.810971 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 8 FOLLOWER]: Advancing to term 9
I20250111 09:01:29.810994 17691 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 7 FOLLOWER]: Leader election won for term 7
I20250111 09:01:29.811076 17691 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 7 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.811182 17691 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 4.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.811651 17476 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.811645 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 6 to 8, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 8 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.812130 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.812218 17340 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 8 LEADER]: Stepping down as leader of term 8
I20250111 09:01:29.812258 17340 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.812369 17340 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 8.3, Last appended by leader: 3, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.812495 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 8 FOLLOWER]: Advancing to term 9
I20250111 09:01:29.812633 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 6 to 7, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 7 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.813166 17340 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 9 because replica has last-logged OpId of term: 8 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20250111 09:01:29.813335 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.813419 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 8 FOLLOWER]: Advancing to term 9
I20250111 09:01:29.814157 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 9.
I20250111 09:01:29.814422 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.814596 17691 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 9 FOLLOWER]: Leader election won for term 9
I20250111 09:01:29.814709 17691 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 9 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.813352 17476 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 9 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.814798 17691 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 6.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.815058 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:29.815460 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:29.815650 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.815696 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 7 FOLLOWER]: Advancing to term 8
I20250111 09:01:29.815891 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 8 to 9, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 9 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.816135 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.816493 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 8 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.816613 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 8 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.816617 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 8 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.816702 17476 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 7 LEADER]: Stepping down as leader of term 7
I20250111 09:01:29.816743 17476 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.816812 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 7 FOLLOWER]: Advancing to term 8
I20250111 09:01:29.816892 17476 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.817196 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:29.817271 17613 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.817049 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 7 FOLLOWER]: Advancing to term 8
I20250111 09:01:29.817329 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 9 FOLLOWER]: Advancing to term 10
I20250111 09:01:29.817792 17340 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 8 because replica has last-logged OpId of term: 6 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.817902 17613 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.817955 17476 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 8 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.818146 17613 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 10 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.818464 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 10 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.818540 17476 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 9 LEADER]: Stepping down as leader of term 9
I20250111 09:01:29.818580 17476 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.818665 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.818696 17476 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.818778 17690 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20250111 09:01:29.818825 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 9 FOLLOWER]: Advancing to term 10
I20250111 09:01:29.818908 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 10 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.818995 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 9 FOLLOWER]: Advancing to term 10
I20250111 09:01:29.819492 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 10 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.819636 17340 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 10 because replica has last-logged OpId of term: 8 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.819653 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:29.819886 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.820003 17690 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20250111 09:01:29.820215 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.820302 17340 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.820353 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 8 FOLLOWER]: Advancing to term 9
I20250111 09:01:29.820802 17340 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.821094 17340 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 9 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.821362 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 9 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.821429 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 9 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.821450 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 8 FOLLOWER]: Advancing to term 9
I20250111 09:01:29.821506 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 8 FOLLOWER]: Advancing to term 9
I20250111 09:01:29.821660 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.821749 17340 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.821803 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 10 FOLLOWER]: Advancing to term 11
I20250111 09:01:29.822109 17476 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 9 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 2.
I20250111 09:01:29.822129 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 9.
I20250111 09:01:29.822325 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:29.822387 17340 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.822616 17699 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 9 FOLLOWER]: Leader election won for term 9
I20250111 09:01:29.822683 17340 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 11 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.822690 17699 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 9 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.822952 17699 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 6.2, Last appended by leader: 2, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.823020 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 11 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.823098 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 10 FOLLOWER]: Advancing to term 11
I20250111 09:01:29.823815 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 11.
I20250111 09:01:29.824091 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:29.824096 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:29.824237 17687 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 11 FOLLOWER]: Leader election won for term 11
I20250111 09:01:29.824332 17687 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 11 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.824375 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 7 to 9, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 9 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.824455 17687 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 8.3, Last appended by leader: 3, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.824170 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 11 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.825026 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 10 FOLLOWER]: Advancing to term 11
I20250111 09:01:29.825026 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.825146 17467 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.825198 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 9 FOLLOWER]: Advancing to term 10
I20250111 09:01:29.825835 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 11 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 3.
I20250111 09:01:29.825861 17467 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.826248 17467 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 10 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.826568 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.826558 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.826646 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 9 FOLLOWER]: Advancing to term 10
I20250111 09:01:29.826646 17340 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 9 LEADER]: Stepping down as leader of term 9
I20250111 09:01:29.826680 17340 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.826812 17340 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 9.3, Last appended by leader: 3, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.826932 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 9 FOLLOWER]: Advancing to term 10
I20250111 09:01:29.826892 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 9 to 11, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 11 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.826891 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.827109 17467 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.827195 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 11 FOLLOWER]: Advancing to term 12
I20250111 09:01:29.827299 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 10.
I20250111 09:01:29.827512 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:29.827656 17691 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 10 FOLLOWER]: Leader election won for term 10
I20250111 09:01:29.827754 17340 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 10 because replica has last-logged OpId of term: 9 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
I20250111 09:01:29.827745 17691 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 10 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.827850 17691 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 7.3, Last appended by leader: 3, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.828200 17467 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.828527 17467 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 12 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.828831 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.828915 17339 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 11 LEADER]: Stepping down as leader of term 11
I20250111 09:01:29.828927 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.828953 17339 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.829031 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 11 FOLLOWER]: Advancing to term 12
I20250111 09:01:29.829092 17339 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 11.4, Last appended by leader: 4, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.829201 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 11 FOLLOWER]: Advancing to term 12
I20250111 09:01:29.829597 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 12.
I20250111 09:01:29.829737 17339 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 12 because replica has last-logged OpId of term: 11 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20250111 09:01:29.829809 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:29.829927 17695 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 12 FOLLOWER]: Leader election won for term 12
I20250111 09:01:29.829998 17695 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 12 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.830085 17695 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 9.4, Last appended by leader: 4, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.831029 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:29.831387 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:29.831483 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.831537 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 10 FOLLOWER]: Advancing to term 11
I20250111 09:01:29.832120 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.832368 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 11 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.832580 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 11 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.832670 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 10 FOLLOWER]: Advancing to term 11
I20250111 09:01:29.832691 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 11 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.832834 17467 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 10 LEADER]: Stepping down as leader of term 10
I20250111 09:01:29.832892 17467 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.833036 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:29.833127 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.833179 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 12 FOLLOWER]: Advancing to term 13
I20250111 09:01:29.833061 17467 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 10.4, Last appended by leader: 4, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.833285 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 10 FOLLOWER]: Advancing to term 11
I20250111 09:01:29.833400 17339 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 11 because replica has last-logged OpId of term: 9 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.833847 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.834142 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 13 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.834352 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 13 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.834472 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 12 FOLLOWER]: Advancing to term 13
I20250111 09:01:29.834498 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 13 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.834591 17476 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 12 LEADER]: Stepping down as leader of term 12
I20250111 09:01:29.834635 17476 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.834764 17476 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.834865 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 12 FOLLOWER]: Advancing to term 13
I20250111 09:01:29.835145 17339 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 13 because replica has last-logged OpId of term: 11 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.835464 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:29.835471 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 13 because replica has last-logged OpId of term: 12 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.835675 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.835820 17339 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.835852 17690 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20250111 09:01:29.835904 17467 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 11 because replica has last-logged OpId of term: 10 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.835907 17339 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.835954 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 11 FOLLOWER]: Advancing to term 12
I20250111 09:01:29.836139 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.836318 17690 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20250111 09:01:29.836525 17339 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.836767 17339 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 12 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.837055 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 12 candidate_status { last_received { term: 9 index: 3 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.837141 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 11 FOLLOWER]: Advancing to term 12
I20250111 09:01:29.837328 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 11 to 12, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 12 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.837797 17476 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 12 because replica has last-logged OpId of term: 10 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 3.
I20250111 09:01:29.838080 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.838167 17338 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.838220 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 13 FOLLOWER]: Advancing to term 14
I20250111 09:01:29.838251 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 12 candidate_status { last_received { term: 9 index: 3 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.838339 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 11 FOLLOWER]: Advancing to term 12
I20250111 09:01:29.838912 17338 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.839046 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 12.
I20250111 09:01:29.839229 17338 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 14 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.839365 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:01:29.839437 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 14 candidate_status { last_received { term: 11 index: 4 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.839515 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 13 FOLLOWER]: Advancing to term 14
I20250111 09:01:29.839540 17687 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 12 FOLLOWER]: Leader election won for term 12
I20250111 09:01:29.839627 17687 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 12 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.839720 17687 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 9.3, Last appended by leader: 3, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.840118 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 14 because replica has last-logged OpId of term: 12 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 4.
I20250111 09:01:29.840276 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:29.840292 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 14 candidate_status { last_received { term: 11 index: 4 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.840371 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 13 FOLLOWER]: Advancing to term 14
I20250111 09:01:29.840624 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.840747 17476 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.840838 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 12 FOLLOWER]: Advancing to term 13
I20250111 09:01:29.840989 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 14.
I20250111 09:01:29.840997 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.841215 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:01:29.841382 17687 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 14 FOLLOWER]: Leader election won for term 14
I20250111 09:01:29.841442 17687 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 14 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.841536 17687 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 11.4, Last appended by leader: 4, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.841557 17476 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.841995 17476 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 13 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.842132 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.842243 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 12 FOLLOWER]: Advancing to term 13
I20250111 09:01:29.842360 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.842446 17340 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 12 LEADER]: Stepping down as leader of term 12
I20250111 09:01:29.842489 17340 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.842617 17340 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 12.4, Last appended by leader: 4, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.842725 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 12 FOLLOWER]: Advancing to term 13
I20250111 09:01:29.843117 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 13.
I20250111 09:01:29.843267 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.843353 17476 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.843428 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 14 FOLLOWER]: Advancing to term 15
I20250111 09:01:29.843433 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:29.843652 17695 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 13 FOLLOWER]: Leader election won for term 13
I20250111 09:01:29.843721 17695 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 13 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.843814 17695 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 10.4, Last appended by leader: 4, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.844112 17476 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.844417 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 12 to 14, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 14 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.844774 17476 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 15 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.844960 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.845081 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 14 FOLLOWER]: Advancing to term 15
I20250111 09:01:29.845114 17180 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 12 to 13, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 13 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.845697 17613 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 15.
I20250111 09:01:29.845939 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:29.846107 17691 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 15 FOLLOWER]: Leader election won for term 15
I20250111 09:01:29.846163 17691 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 15 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.846244 17691 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 12.5, Last appended by leader: 5, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.846684 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:29.847174 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:29.847266 17613 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.847313 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 13 FOLLOWER]: Advancing to term 14
I20250111 09:01:29.847349 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.847424 17338 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 14 LEADER]: Stepping down as leader of term 14
I20250111 09:01:29.847451 17338 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.847556 17338 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 14.5, Last appended by leader: 5, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.847642 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 14 FOLLOWER]: Advancing to term 15
I20250111 09:01:29.847918 17613 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.848166 17613 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 14 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.848184 17338 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 15 because replica has last-logged OpId of term: 14 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 5.
I20250111 09:01:29.848520 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 14 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.848613 17476 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 13 LEADER]: Stepping down as leader of term 13
I20250111 09:01:29.848659 17476 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.848783 17476 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 13.5, Last appended by leader: 5, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.848912 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 13 FOLLOWER]: Advancing to term 14
I20250111 09:01:29.848912 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:29.848997 17613 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.849099 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 15 FOLLOWER]: Advancing to term 16
I20250111 09:01:29.849695 17613 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.849749 17476 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 14 because replica has last-logged OpId of term: 13 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.849792 17180 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 14 to 15, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 15 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.850015 17613 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 16 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.850308 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 16 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.850411 17476 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 15 LEADER]: Stepping down as leader of term 15
I20250111 09:01:29.850459 17476 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.850452 17340 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 13 because replica has last-logged OpId of term: 12 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 4.
I20250111 09:01:29.850576 17476 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 15.6, Last appended by leader: 6, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.850688 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 15 FOLLOWER]: Advancing to term 16
I20250111 09:01:29.851073 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 14 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.851161 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 13 FOLLOWER]: Advancing to term 14
I20250111 09:01:29.851344 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 16 because replica has last-logged OpId of term: 15 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.851780 17338 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 14 because replica has last-logged OpId of term: 12 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.851995 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.852187 17690 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20250111 09:01:29.852344 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:29.852456 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 16 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.852535 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 15 FOLLOWER]: Advancing to term 16
I20250111 09:01:29.852686 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.852785 17338 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.852836 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 14 FOLLOWER]: Advancing to term 15
I20250111 09:01:29.852990 17339 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 16 because replica has last-logged OpId of term: 14 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.853263 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.853466 17690 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20250111 09:01:29.853484 17338 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.853782 17338 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 15 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.854075 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 15 candidate_status { last_received { term: 12 index: 4 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.854120 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 15 candidate_status { last_received { term: 12 index: 4 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.854173 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 14 FOLLOWER]: Advancing to term 15
I20250111 09:01:29.854218 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 14 FOLLOWER]: Advancing to term 15
I20250111 09:01:29.854367 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.854461 17338 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.854507 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 16 FOLLOWER]: Advancing to term 17
I20250111 09:01:29.854920 17613 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 15.
I20250111 09:01:29.854965 17476 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 15 because replica has last-logged OpId of term: 13 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 4.
I20250111 09:01:29.855199 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:01:29.855265 17338 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.855388 17699 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 15 FOLLOWER]: Leader election won for term 15
I20250111 09:01:29.855458 17699 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 15 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.855510 17338 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 17 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.855554 17699 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 12.4, Last appended by leader: 4, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.856387 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 17 candidate_status { last_received { term: 14 index: 5 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.856421 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 17 candidate_status { last_received { term: 14 index: 5 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.856475 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 16 FOLLOWER]: Advancing to term 17
I20250111 09:01:29.856488 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 16 FOLLOWER]: Advancing to term 17
I20250111 09:01:29.856554 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 13 to 15, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 15 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.857095 17613 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 17.
I20250111 09:01:29.857144 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 17 because replica has last-logged OpId of term: 15 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 5.
I20250111 09:01:29.857307 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:29.857440 17699 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 17 FOLLOWER]: Leader election won for term 17
I20250111 09:01:29.857501 17699 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 17 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.857590 17699 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 14.5, Last appended by leader: 5, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.857791 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:29.858182 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.858294 17476 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.858350 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 15 FOLLOWER]: Advancing to term 16
I20250111 09:01:29.858997 17476 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.859124 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 15 to 17, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 17 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.859333 17476 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 16 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.859577 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 16 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.859639 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 16 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.859666 17338 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 15 LEADER]: Stepping down as leader of term 15
I20250111 09:01:29.859704 17338 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.859723 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 15 FOLLOWER]: Advancing to term 16
I20250111 09:01:29.859845 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.859858 17338 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 15.5, Last appended by leader: 5, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.859934 17476 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.860001 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 17 FOLLOWER]: Advancing to term 18
I20250111 09:01:29.859997 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 15 FOLLOWER]: Advancing to term 16
I20250111 09:01:29.860457 17476 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.860946 17338 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 16 because replica has last-logged OpId of term: 15 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 5.
I20250111 09:01:29.861092 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 18 candidate_status { last_received { term: 15 index: 6 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.861172 17613 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 16.
I20250111 09:01:29.861196 17476 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 18 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.861361 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 18 candidate_status { last_received { term: 15 index: 6 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.861379 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.861441 17339 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 17 LEADER]: Stepping down as leader of term 17
I20250111 09:01:29.861474 17339 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.861529 17695 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 16 FOLLOWER]: Leader election won for term 16
I20250111 09:01:29.861599 17695 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 16 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.861588 17339 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 17.6, Last appended by leader: 6, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.861694 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 17 FOLLOWER]: Advancing to term 18
I20250111 09:01:29.861690 17695 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 13.5, Last appended by leader: 5, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.862318 17339 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 18 because replica has last-logged OpId of term: 17 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 6.
I20250111 09:01:29.861178 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 17 FOLLOWER]: Advancing to term 18
I20250111 09:01:29.862774 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 15 to 16, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 16 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.862978 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 18.
I20250111 09:01:29.863220 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.863437 17691 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 18 FOLLOWER]: Leader election won for term 18
I20250111 09:01:29.863536 17691 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 18 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.863639 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:29.863648 17691 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 15.6, Last appended by leader: 6, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.864256 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:29.864365 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.864429 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 16 FOLLOWER]: Advancing to term 17
I20250111 09:01:29.864969 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.865161 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 17 to 18, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 18 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.865307 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 17 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.865566 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 17 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.865657 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 16 FOLLOWER]: Advancing to term 17
I20250111 09:01:29.865918 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:29.865947 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 17 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.866008 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.866058 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 18 FOLLOWER]: Advancing to term 19
I20250111 09:01:29.866138 17476 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 16 LEADER]: Stepping down as leader of term 16
I20250111 09:01:29.866214 17476 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.866355 17339 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 17 because replica has last-logged OpId of term: 15 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.866372 17476 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 16.6, Last appended by leader: 6, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.866531 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 16 FOLLOWER]: Advancing to term 17
I20250111 09:01:29.866875 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.867107 17476 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 17 because replica has last-logged OpId of term: 16 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.867349 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 19 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.867398 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.867434 17476 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 18 LEADER]: Stepping down as leader of term 18
I20250111 09:01:29.867530 17690 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20250111 09:01:29.867515 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 19 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.867468 17476 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.867589 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 18 FOLLOWER]: Advancing to term 19
I20250111 09:01:29.867688 17476 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 18.7, Last appended by leader: 7, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.867784 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 18 FOLLOWER]: Advancing to term 19
I20250111 09:01:29.868081 17339 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 19 because replica has last-logged OpId of term: 17 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.868194 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 19 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.868350 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 19 because replica has last-logged OpId of term: 18 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.868602 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.868726 17690 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20250111 09:01:29.870514 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:29.870872 17339 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.870947 17339 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.870998 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 17 FOLLOWER]: Advancing to term 18
I20250111 09:01:29.871445 17339 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.871665 17339 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 18 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.871944 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 18 candidate_status { last_received { term: 15 index: 5 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.871944 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 18 candidate_status { last_received { term: 15 index: 5 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.872031 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 17 FOLLOWER]: Advancing to term 18
I20250111 09:01:29.872031 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 17 FOLLOWER]: Advancing to term 18
I20250111 09:01:29.872167 17339 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.872253 17339 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.872306 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 19 FOLLOWER]: Advancing to term 20
I20250111 09:01:29.872624 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 18.
I20250111 09:01:29.872666 17476 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 18 because replica has last-logged OpId of term: 16 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 5.
I20250111 09:01:29.872788 17339 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.872915 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:01:29.873086 17699 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 18 FOLLOWER]: Leader election won for term 18
I20250111 09:01:29.873088 17339 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 20 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.873188 17699 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 18 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.873292 17699 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 15.5, Last appended by leader: 5, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.873451 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:29.873481 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 20 candidate_status { last_received { term: 17 index: 6 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.873651 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 19 FOLLOWER]: Advancing to term 20
I20250111 09:01:29.873864 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.873965 17467 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.874029 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 18 FOLLOWER]: Advancing to term 19
I20250111 09:01:29.874037 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 20 candidate_status { last_received { term: 17 index: 6 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.874141 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 19 FOLLOWER]: Advancing to term 20
I20250111 09:01:29.874508 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 16 to 18, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 18 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.874758 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 20 because replica has last-logged OpId of term: 18 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 6.
I20250111 09:01:29.874814 17467 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.874949 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 20.
I20250111 09:01:29.875087 17467 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 19 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.875121 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:01:29.875308 17699 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 20 FOLLOWER]: Leader election won for term 20
I20250111 09:01:29.875375 17699 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 20 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.875419 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 19 candidate_status { last_received { term: 16 index: 6 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.875488 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 18 FOLLOWER]: Advancing to term 19
I20250111 09:01:29.875474 17699 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 17.6, Last appended by leader: 6, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.875844 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.875924 17467 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.875967 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 20 FOLLOWER]: Advancing to term 21
I20250111 09:01:29.876137 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 19.
I20250111 09:01:29.876338 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:29.876482 17695 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 19 FOLLOWER]: Leader election won for term 19
I20250111 09:01:29.876566 17695 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 19 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.876652 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 19 candidate_status { last_received { term: 16 index: 6 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.876688 17695 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 16.6, Last appended by leader: 6, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.876747 17339 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 18 LEADER]: Stepping down as leader of term 18
I20250111 09:01:29.876803 17339 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.876890 17339 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 18.6, Last appended by leader: 6, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.876987 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 18 FOLLOWER]: Advancing to term 19
I20250111 09:01:29.876582 17467 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.877396 17467 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 21 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.877696 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 21 candidate_status { last_received { term: 18 index: 7 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.877718 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 18 to 20, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 20 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.877801 17338 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 20 LEADER]: Stepping down as leader of term 20
I20250111 09:01:29.877835 17338 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.877933 17338 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 20.7, Last appended by leader: 7, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.877909 17180 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 18 to 19, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 19 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.878036 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 20 FOLLOWER]: Advancing to term 21
I20250111 09:01:29.878398 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 21 candidate_status { last_received { term: 18 index: 7 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.878482 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 20 FOLLOWER]: Advancing to term 21
I20250111 09:01:29.878597 17338 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 21 because replica has last-logged OpId of term: 20 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 7.
I20250111 09:01:29.879112 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 21.
I20250111 09:01:29.879184 17339 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 19 because replica has last-logged OpId of term: 18 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 6.
I20250111 09:01:29.879439 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.879603 17695 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 21 FOLLOWER]: Leader election won for term 21
I20250111 09:01:29.879673 17695 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 21 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.879765 17695 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 18.7, Last appended by leader: 7, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.880910 17180 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 20 to 21, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 21 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.883476 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:29.883896 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:29.883994 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.884055 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 19 FOLLOWER]: Advancing to term 20
I20250111 09:01:29.884589 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.884886 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 20 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.885182 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 20 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.885280 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 19 FOLLOWER]: Advancing to term 20
I20250111 09:01:29.885406 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 20 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.885495 17467 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 19 LEADER]: Stepping down as leader of term 19
I20250111 09:01:29.885537 17467 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.885649 17467 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 19.7, Last appended by leader: 7, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.885754 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 19 FOLLOWER]: Advancing to term 20
I20250111 09:01:29.885984 17339 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 20 because replica has last-logged OpId of term: 18 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.886117 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:29.886193 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.886237 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 21 FOLLOWER]: Advancing to term 22
I20250111 09:01:29.886361 17467 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 20 because replica has last-logged OpId of term: 19 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.886636 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.886830 17690 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20250111 09:01:29.886843 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.887306 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 22 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.887728 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 22 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.887737 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 22 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.887809 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 21 FOLLOWER]: Advancing to term 22
I20250111 09:01:29.887816 17467 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 21 LEADER]: Stepping down as leader of term 21
I20250111 09:01:29.887852 17467 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.887964 17467 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 21.8, Last appended by leader: 8, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.888043 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 21 FOLLOWER]: Advancing to term 22
I20250111 09:01:29.888391 17339 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 22 because replica has last-logged OpId of term: 20 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.888582 17467 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 22 because replica has last-logged OpId of term: 21 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.888833 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.888959 17690 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20250111 09:01:29.890682 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:29.891112 17339 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.891191 17339 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.891229 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 20 FOLLOWER]: Advancing to term 21
I20250111 09:01:29.891678 17339 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.891916 17339 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 21 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.892200 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 21 candidate_status { last_received { term: 18 index: 6 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.892187 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 21 candidate_status { last_received { term: 18 index: 6 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.892277 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 20 FOLLOWER]: Advancing to term 21
I20250111 09:01:29.892277 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 20 FOLLOWER]: Advancing to term 21
I20250111 09:01:29.892439 17339 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.892531 17339 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.892582 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 22 FOLLOWER]: Advancing to term 23
I20250111 09:01:29.892903 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 21.
I20250111 09:01:29.892925 17467 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 21 because replica has last-logged OpId of term: 19 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 6.
I20250111 09:01:29.893229 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:01:29.893380 17339 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.893414 17687 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 21 FOLLOWER]: Leader election won for term 21
I20250111 09:01:29.893546 17687 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 21 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.893666 17687 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 18.6, Last appended by leader: 6, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.893960 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 23 candidate_status { last_received { term: 20 index: 7 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.894065 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 22 FOLLOWER]: Advancing to term 23
I20250111 09:01:29.894702 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 23 candidate_status { last_received { term: 20 index: 7 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.894793 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 22 FOLLOWER]: Advancing to term 23
I20250111 09:01:29.895023 17467 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 23 because replica has last-logged OpId of term: 21 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 7.
I20250111 09:01:29.895023 17180 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 19 to 21, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 21 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.893739 17339 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 23 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.895684 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 23.
I20250111 09:01:29.896059 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:01:29.896250 17699 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 23 FOLLOWER]: Leader election won for term 23
I20250111 09:01:29.896324 17699 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 23 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.896411 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:29.896441 17699 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 20.7, Last appended by leader: 7, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.896852 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.896929 17467 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.896968 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 21 FOLLOWER]: Advancing to term 22
I20250111 09:01:29.897572 17467 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.897614 17180 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 21 to 23, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 23 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.897815 17467 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 22 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.898139 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 22 candidate_status { last_received { term: 19 index: 7 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.898172 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 22 candidate_status { last_received { term: 19 index: 7 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.898233 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 21 FOLLOWER]: Advancing to term 22
I20250111 09:01:29.898259 17339 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 21 LEADER]: Stepping down as leader of term 21
I20250111 09:01:29.898306 17339 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.898342 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.898452 17467 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.898447 17339 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 21.7, Last appended by leader: 7, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.898506 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 23 FOLLOWER]: Advancing to term 24
I20250111 09:01:29.898558 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 21 FOLLOWER]: Advancing to term 22
I20250111 09:01:29.899161 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 22.
I20250111 09:01:29.899163 17467 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.899295 17339 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 22 because replica has last-logged OpId of term: 21 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 7.
I20250111 09:01:29.899389 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:29.899443 17467 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 24 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.899662 17691 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 22 FOLLOWER]: Leader election won for term 22
I20250111 09:01:29.899741 17691 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 22 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.899839 17691 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 19.7, Last appended by leader: 7, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.899946 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 24 candidate_status { last_received { term: 21 index: 8 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.900024 17339 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 23 LEADER]: Stepping down as leader of term 23
I20250111 09:01:29.900064 17339 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.900182 17339 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 23.8, Last appended by leader: 8, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.900310 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 23 FOLLOWER]: Advancing to term 24
I20250111 09:01:29.900959 17339 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 24 because replica has last-logged OpId of term: 23 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 8.
I20250111 09:01:29.901202 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 24 candidate_status { last_received { term: 21 index: 8 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.901288 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 23 FOLLOWER]: Advancing to term 24
I20250111 09:01:29.901926 17613 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 24.
I20250111 09:01:29.902487 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.902662 17691 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 24 FOLLOWER]: Leader election won for term 24
I20250111 09:01:29.902742 17691 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 24 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.902849 17691 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 21.8, Last appended by leader: 8, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.903185 17181 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54466:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20250111 09:01:29.903631 17181 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250111 09:01:29.903574 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 21 to 22, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 22 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.906230 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:29.906601 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:29.906685 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.906720 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 22 FOLLOWER]: Advancing to term 23
I20250111 09:01:29.906637 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 23 to 24, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 24 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.907325 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.907598 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 23 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.908008 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 23 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.908092 17467 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 22 LEADER]: Stepping down as leader of term 22
I20250111 09:01:29.908121 17467 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.907876 17320 tablet_service.cc:1467] Processing CreateTablet for tablet 2f60a4d7104242869ce22b7a9c250295 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=30ccf514e62a4b908cbe6b373242d14c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:01:29.908216 17467 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 22.8, Last appended by leader: 8, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.908303 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 22 FOLLOWER]: Advancing to term 23
I20250111 09:01:29.908406 17320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f60a4d7104242869ce22b7a9c250295. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:29.908179 17457 tablet_service.cc:1467] Processing CreateTablet for tablet 2f60a4d7104242869ce22b7a9c250295 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=30ccf514e62a4b908cbe6b373242d14c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:01:29.908205 17593 tablet_service.cc:1467] Processing CreateTablet for tablet 2f60a4d7104242869ce22b7a9c250295 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=30ccf514e62a4b908cbe6b373242d14c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:01:29.908633 17457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f60a4d7104242869ce22b7a9c250295. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:29.908812 17593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f60a4d7104242869ce22b7a9c250295. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:01:29.908912 17467 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 23 because replica has last-logged OpId of term: 22 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.909157 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:29.909236 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 24 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.909277 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 24 FOLLOWER]: Advancing to term 25
I20250111 09:01:29.909924 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 25 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.910236 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 25 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.910853 17686 tablet_bootstrap.cc:492] T 2f60a4d7104242869ce22b7a9c250295 P 262623ac6df745c785ce267ea39bd265: Bootstrap starting.
I20250111 09:01:29.910992 17683 tablet_bootstrap.cc:492] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c: Bootstrap starting.
I20250111 09:01:29.911455 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 25 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.911535 17467 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 24 LEADER]: Stepping down as leader of term 24
I20250111 09:01:29.911569 17467 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.911682 17467 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 24.9, Last appended by leader: 9, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.911796 17683 tablet_bootstrap.cc:654] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:29.911787 17686 tablet_bootstrap.cc:654] T 2f60a4d7104242869ce22b7a9c250295 P 262623ac6df745c785ce267ea39bd265: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:29.911841 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 25 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.911918 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 24 FOLLOWER]: Advancing to term 25
I20250111 09:01:29.912323 17682 tablet_bootstrap.cc:492] T 2f60a4d7104242869ce22b7a9c250295 P fb270e42d7c9400f8c05bb9565db1972: Bootstrap starting.
I20250111 09:01:29.912616 17338 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 25 because replica has last-logged OpId of term: 23 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.912719 17686 tablet_bootstrap.cc:492] T 2f60a4d7104242869ce22b7a9c250295 P 262623ac6df745c785ce267ea39bd265: No bootstrap required, opened a new log
I20250111 09:01:29.912783 17686 ts_tablet_manager.cc:1397] T 2f60a4d7104242869ce22b7a9c250295 P 262623ac6df745c785ce267ea39bd265: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:01:29.911784 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 24 FOLLOWER]: Advancing to term 25
I20250111 09:01:29.913161 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 23 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.913208 17682 tablet_bootstrap.cc:654] T 2f60a4d7104242869ce22b7a9c250295 P fb270e42d7c9400f8c05bb9565db1972: Neither blocks nor log segments found. Creating new log.
I20250111 09:01:29.913241 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 22 FOLLOWER]: Advancing to term 23
I20250111 09:01:29.913206 17686 raft_consensus.cc:357] T 2f60a4d7104242869ce22b7a9c250295 P 262623ac6df745c785ce267ea39bd265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.913278 17686 raft_consensus.cc:383] T 2f60a4d7104242869ce22b7a9c250295 P 262623ac6df745c785ce267ea39bd265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:29.913306 17686 raft_consensus.cc:738] T 2f60a4d7104242869ce22b7a9c250295 P 262623ac6df745c785ce267ea39bd265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Initialized, Role: FOLLOWER
I20250111 09:01:29.913414 17686 consensus_queue.cc:260] T 2f60a4d7104242869ce22b7a9c250295 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.913663 17686 ts_tablet_manager.cc:1428] T 2f60a4d7104242869ce22b7a9c250295 P 262623ac6df745c785ce267ea39bd265: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:01:29.913869 17467 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 25 because replica has last-logged OpId of term: 24 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.913961 17339 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 23 because replica has last-logged OpId of term: 21 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.914197 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.914366 17690 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20250111 09:01:29.914430 17682 tablet_bootstrap.cc:492] T 2f60a4d7104242869ce22b7a9c250295 P fb270e42d7c9400f8c05bb9565db1972: No bootstrap required, opened a new log
I20250111 09:01:29.914503 17682 ts_tablet_manager.cc:1397] T 2f60a4d7104242869ce22b7a9c250295 P fb270e42d7c9400f8c05bb9565db1972: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:01:29.914484 17691 tablet_replica.cc:440] T 2f60a4d7104242869ce22b7a9c250295 P 262623ac6df745c785ce267ea39bd265: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
I20250111 09:01:29.914606 17691 tablet_replica.cc:443] T 2f60a4d7104242869ce22b7a9c250295 P 262623ac6df745c785ce267ea39bd265: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:29.914949 17682 raft_consensus.cc:357] T 2f60a4d7104242869ce22b7a9c250295 P fb270e42d7c9400f8c05bb9565db1972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.915028 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.915051 17682 raft_consensus.cc:383] T 2f60a4d7104242869ce22b7a9c250295 P fb270e42d7c9400f8c05bb9565db1972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:29.915084 17682 raft_consensus.cc:738] T 2f60a4d7104242869ce22b7a9c250295 P fb270e42d7c9400f8c05bb9565db1972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Initialized, Role: FOLLOWER
I20250111 09:01:29.915192 17690 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20250111 09:01:29.915176 17682 consensus_queue.cc:260] T 2f60a4d7104242869ce22b7a9c250295 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.915233 17683 tablet_bootstrap.cc:492] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c: No bootstrap required, opened a new log
I20250111 09:01:29.915289 17683 ts_tablet_manager.cc:1397] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250111 09:01:29.915403 17682 ts_tablet_manager.cc:1428] T 2f60a4d7104242869ce22b7a9c250295 P fb270e42d7c9400f8c05bb9565db1972: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:01:29.915575 17699 tablet_replica.cc:440] T 2f60a4d7104242869ce22b7a9c250295 P fb270e42d7c9400f8c05bb9565db1972: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
I20250111 09:01:29.915669 17699 tablet_replica.cc:443] T 2f60a4d7104242869ce22b7a9c250295 P fb270e42d7c9400f8c05bb9565db1972: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:29.915795 17683 raft_consensus.cc:357] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.915876 17683 raft_consensus.cc:383] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:01:29.915902 17683 raft_consensus.cc:738] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Initialized, Role: FOLLOWER
I20250111 09:01:29.915978 17683 consensus_queue.cc:260] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.916188 17683 ts_tablet_manager.cc:1428] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:01:29.916345 17690 tablet_replica.cc:440] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
I20250111 09:01:29.916430 17690 tablet_replica.cc:443] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:29.919940 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:29.920333 17339 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.920423 17339 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.920473 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 23 FOLLOWER]: Advancing to term 24
I20250111 09:01:29.920957 17339 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.921234 17339 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 24 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.921473 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 24 candidate_status { last_received { term: 21 index: 7 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.921525 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 24 candidate_status { last_received { term: 21 index: 7 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.921554 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 23 FOLLOWER]: Advancing to term 24
I20250111 09:01:29.921597 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 23 FOLLOWER]: Advancing to term 24
I20250111 09:01:29.921865 17339 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.921957 17339 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 25 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.922013 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 25 FOLLOWER]: Advancing to term 26
I20250111 09:01:29.922158 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 24.
I20250111 09:01:29.922241 17467 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 24 because replica has last-logged OpId of term: 22 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 7.
I20250111 09:01:29.922381 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:29.922510 17699 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 24 FOLLOWER]: Leader election won for term 24
I20250111 09:01:29.922621 17699 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 24 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.922657 17339 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 26 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.922724 17699 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 21.7, Last appended by leader: 7, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.923000 17339 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 26 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.923569 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 26 candidate_status { last_received { term: 23 index: 8 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.923663 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 25 FOLLOWER]: Advancing to term 26
I20250111 09:01:29.923837 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 26 candidate_status { last_received { term: 23 index: 8 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.923929 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 25 FOLLOWER]: Advancing to term 26
I20250111 09:01:29.924083 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 22 to 24, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 24 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.924410 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 26.
I20250111 09:01:29.924706 17467 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 26 because replica has last-logged OpId of term: 24 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 8.
I20250111 09:01:29.924716 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:29.924852 17687 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 26 FOLLOWER]: Leader election won for term 26
I20250111 09:01:29.924934 17687 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 26 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.925066 17687 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 23.8, Last appended by leader: 8, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.926425 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 24 to 26, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 26 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.930781 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:29.931152 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.931263 17467 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 24 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.931324 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 24 FOLLOWER]: Advancing to term 25
I20250111 09:01:29.931851 17467 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 25 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.932087 17467 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 25 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.932379 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 25 candidate_status { last_received { term: 22 index: 8 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.932379 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 25 candidate_status { last_received { term: 22 index: 8 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.932466 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 24 FOLLOWER]: Advancing to term 25
I20250111 09:01:29.932466 17339 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 24 LEADER]: Stepping down as leader of term 24
I20250111 09:01:29.932504 17339 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.932521 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.932611 17467 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 26 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.932655 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 26 FOLLOWER]: Advancing to term 27
I20250111 09:01:29.932623 17339 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 24.8, Last appended by leader: 8, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.932719 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 24 FOLLOWER]: Advancing to term 25
I20250111 09:01:29.933131 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 25.
I20250111 09:01:29.933362 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:29.933425 17339 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 25 because replica has last-logged OpId of term: 24 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 8.
I20250111 09:01:29.933547 17691 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 25 FOLLOWER]: Leader election won for term 25
I20250111 09:01:29.933585 17467 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 27 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.933630 17691 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 25 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.933765 17691 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 22.8, Last appended by leader: 8, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.934600 17467 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 27 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.934911 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 27 candidate_status { last_received { term: 24 index: 9 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.934995 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 26 FOLLOWER]: Advancing to term 27
I20250111 09:01:29.935004 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 24 to 25, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 25 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.934664 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 27 candidate_status { last_received { term: 24 index: 9 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.935377 17339 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 26 LEADER]: Stepping down as leader of term 26
I20250111 09:01:29.935405 17339 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.935504 17339 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 26.9, Last appended by leader: 9, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.935528 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 27.
I20250111 09:01:29.935590 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 26 FOLLOWER]: Advancing to term 27
I20250111 09:01:29.935951 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:29.936077 17691 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 27 FOLLOWER]: Leader election won for term 27
I20250111 09:01:29.936169 17691 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 27 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.936261 17691 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 24.9, Last appended by leader: 9, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.936192 17339 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 27 because replica has last-logged OpId of term: 26 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 9.
I20250111 09:01:29.937341 17180 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 26 to 27, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 27 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.940119 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:29.940465 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:29.940554 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 25 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.940609 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 25 FOLLOWER]: Advancing to term 26
I20250111 09:01:29.941123 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 26 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.941359 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 26 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.941608 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 26 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.941601 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 26 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.941689 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 25 FOLLOWER]: Advancing to term 26
I20250111 09:01:29.941689 17467 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 25 LEADER]: Stepping down as leader of term 25
I20250111 09:01:29.941730 17467 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.941843 17467 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 25.9, Last appended by leader: 9, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.941895 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:29.941946 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 25 FOLLOWER]: Advancing to term 26
I20250111 09:01:29.941987 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 27 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.942041 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 27 FOLLOWER]: Advancing to term 28
I20250111 09:01:29.942358 17339 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 26 because replica has last-logged OpId of term: 24 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.942643 17467 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 26 because replica has last-logged OpId of term: 25 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.942642 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 28 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.942858 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.942921 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 28 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.943017 17690 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20250111 09:01:29.943264 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 28 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.943337 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:29.943349 17476 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 27 LEADER]: Stepping down as leader of term 27
I20250111 09:01:29.943322 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 28 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.943384 17476 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.943444 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 27 FOLLOWER]: Advancing to term 28
I20250111 09:01:29.943527 17476 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 27.10, Last appended by leader: 10, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.943639 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 27 FOLLOWER]: Advancing to term 28
I20250111 09:01:29.944126 17339 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 28 because replica has last-logged OpId of term: 26 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.944307 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 28 because replica has last-logged OpId of term: 27 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.944788 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.944846 17339 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.944941 17339 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 26 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.944941 17690 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20250111 09:01:29.944994 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 26 FOLLOWER]: Advancing to term 27
I20250111 09:01:29.945633 17339 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 27 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.945959 17339 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 27 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.946223 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 27 candidate_status { last_received { term: 24 index: 8 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.946311 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 26 FOLLOWER]: Advancing to term 27
I20250111 09:01:29.946302 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 27 candidate_status { last_received { term: 24 index: 8 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.946381 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 26 FOLLOWER]: Advancing to term 27
I20250111 09:01:29.946851 17476 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 27 because replica has last-logged OpId of term: 25 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 8.
I20250111 09:01:29.946941 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 27.
I20250111 09:01:29.947145 17339 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.947186 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:01:29.947243 17339 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 28 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.947296 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 28 FOLLOWER]: Advancing to term 29
I20250111 09:01:29.947384 17687 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 27 FOLLOWER]: Leader election won for term 27
I20250111 09:01:29.947458 17687 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 27 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.947571 17687 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 24.8, Last appended by leader: 8, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.947927 17339 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 29 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.948202 17339 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 29 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.948434 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 29 candidate_status { last_received { term: 26 index: 9 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.948519 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 28 FOLLOWER]: Advancing to term 29
I20250111 09:01:29.948529 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 29 candidate_status { last_received { term: 26 index: 9 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.948596 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 28 FOLLOWER]: Advancing to term 29
I20250111 09:01:29.949178 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 29.
I20250111 09:01:29.949220 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 29 because replica has last-logged OpId of term: 27 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 9.
I20250111 09:01:29.949388 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:29.949470 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 25 to 27, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 27 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.949641 17699 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 29 FOLLOWER]: Leader election won for term 29
I20250111 09:01:29.949755 17699 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 29 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.949875 17699 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 26.9, Last appended by leader: 9, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.951714 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 27 to 29, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 29 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.955575 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:29.955937 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.956019 17476 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 27 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.956072 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 27 FOLLOWER]: Advancing to term 28
I20250111 09:01:29.956529 17476 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 28 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.956761 17476 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 28 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.956986 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 28 candidate_status { last_received { term: 25 index: 9 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.957109 17339 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 27 LEADER]: Stepping down as leader of term 27
I20250111 09:01:29.957122 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 28 candidate_status { last_received { term: 25 index: 9 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.957151 17339 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.957201 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 27 FOLLOWER]: Advancing to term 28
I20250111 09:01:29.957281 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.957262 17339 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 27.9, Last appended by leader: 9, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.957356 17476 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 29 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.957391 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 29 FOLLOWER]: Advancing to term 30
I20250111 09:01:29.957376 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 27 FOLLOWER]: Advancing to term 28
I20250111 09:01:29.957913 17339 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 28 because replica has last-logged OpId of term: 27 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 9.
I20250111 09:01:29.957950 17476 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 30 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.958250 17476 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 30 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.958443 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 30 candidate_status { last_received { term: 27 index: 10 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.958526 17339 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 29 LEADER]: Stepping down as leader of term 29
I20250111 09:01:29.958562 17339 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.958580 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 30 candidate_status { last_received { term: 27 index: 10 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.958668 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 29 FOLLOWER]: Advancing to term 30
I20250111 09:01:29.958669 17339 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 29.10, Last appended by leader: 10, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.958777 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 29 FOLLOWER]: Advancing to term 30
I20250111 09:01:29.958981 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 28.
I20250111 09:01:29.959190 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.959275 17613 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 30.
I20250111 09:01:29.959331 17691 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 28 FOLLOWER]: Leader election won for term 28
I20250111 09:01:29.959396 17691 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 28 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.959455 17339 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 30 because replica has last-logged OpId of term: 29 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 10.
I20250111 09:01:29.959486 17691 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 25.9, Last appended by leader: 9, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.959713 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.959838 17695 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 30 FOLLOWER]: Leader election won for term 30
I20250111 09:01:29.959905 17695 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 30 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.960006 17695 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 27.10, Last appended by leader: 10, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.960549 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 27 to 28, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 28 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.962653 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 29 to 30, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 30 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.966518 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:29.966885 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:29.966972 17613 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 28 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.967031 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 28 FOLLOWER]: Advancing to term 29
I20250111 09:01:29.967520 17613 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 29 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.967775 17613 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 29 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.968000 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 29 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.968061 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 29 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.968091 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 28 FOLLOWER]: Advancing to term 29
I20250111 09:01:29.968150 17476 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 28 LEADER]: Stepping down as leader of term 28
I20250111 09:01:29.968194 17476 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.968308 17476 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 28.10, Last appended by leader: 10, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.968348 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:29.968420 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 28 FOLLOWER]: Advancing to term 29
I20250111 09:01:29.968427 17613 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 30 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.968468 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 30 FOLLOWER]: Advancing to term 31
I20250111 09:01:29.968756 17339 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 29 because replica has last-logged OpId of term: 27 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.968963 17476 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 29 because replica has last-logged OpId of term: 28 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.969123 17613 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 31 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.969208 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.969355 17690 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20250111 09:01:29.969398 17613 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 31 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.969630 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 31 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.969744 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 30 FOLLOWER]: Advancing to term 31
I20250111 09:01:29.969844 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 31 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.969946 17476 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 30 LEADER]: Stepping down as leader of term 30
I20250111 09:01:29.969990 17476 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.970113 17476 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 30.11, Last appended by leader: 11, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.970216 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 30 FOLLOWER]: Advancing to term 31
I20250111 09:01:29.970434 17339 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 31 because replica has last-logged OpId of term: 29 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.970875 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 31 because replica has last-logged OpId of term: 30 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:29.971110 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.971311 17690 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20250111 09:01:29.973767 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:29.974160 17339 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.974246 17339 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 29 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.974287 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 29 FOLLOWER]: Advancing to term 30
I20250111 09:01:29.974812 17339 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 30 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.975059 17339 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 30 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.975379 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 30 candidate_status { last_received { term: 27 index: 9 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.975472 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 29 FOLLOWER]: Advancing to term 30
I20250111 09:01:29.975438 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 30 candidate_status { last_received { term: 27 index: 9 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.975541 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 29 FOLLOWER]: Advancing to term 30
I20250111 09:01:29.975641 17339 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:29.975786 17690 raft_consensus.cc:491] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:01:29.975833 17339 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 31 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.975848 17690 raft_consensus.cc:3054] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:29.975888 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 31 FOLLOWER]: Advancing to term 32
I20250111 09:01:29.976140 17476 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 30 because replica has last-logged OpId of term: 28 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 9.
I20250111 09:01:29.976346 17613 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 30.
I20250111 09:01:29.976634 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:01:29.976675 17339 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 32 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.976758 17690 raft_consensus.cc:513] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.976878 17699 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 30 FOLLOWER]: Leader election won for term 30
I20250111 09:01:29.976971 17699 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 30 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.977011 17339 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 32 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.977186 17690 leader_election.cc:290] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 1 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:29.977180 17699 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 27.9, Last appended by leader: 9, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.977610 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f60a4d7104242869ce22b7a9c250295" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.977715 17476 raft_consensus.cc:3054] T 2f60a4d7104242869ce22b7a9c250295 P 262623ac6df745c785ce267ea39bd265 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:29.978288 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f60a4d7104242869ce22b7a9c250295" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.978379 17339 raft_consensus.cc:3054] T 2f60a4d7104242869ce22b7a9c250295 P fb270e42d7c9400f8c05bb9565db1972 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:01:29.978379 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 32 candidate_status { last_received { term: 29 index: 10 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.978389 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 32 candidate_status { last_received { term: 29 index: 10 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:29.978477 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 31 FOLLOWER]: Advancing to term 32
I20250111 09:01:29.978484 17476 raft_consensus.cc:2463] T 2f60a4d7104242869ce22b7a9c250295 P 262623ac6df745c785ce267ea39bd265 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 1.
I20250111 09:01:29.978484 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 31 FOLLOWER]: Advancing to term 32
I20250111 09:01:29.978770 17545 leader_election.cc:304] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:29.978916 17690 raft_consensus.cc:2798] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:01:29.979061 17690 raft_consensus.cc:695] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c [term 1 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:29.979140 17613 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 32.
I20250111 09:01:29.979166 17339 raft_consensus.cc:2463] T 2f60a4d7104242869ce22b7a9c250295 P fb270e42d7c9400f8c05bb9565db1972 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 1.
I20250111 09:01:29.979431 17467 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 32 because replica has last-logged OpId of term: 30 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 10.
I20250111 09:01:29.979390 17690 consensus_queue.cc:237] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.979454 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:29.979593 17699 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 32 FOLLOWER]: Leader election won for term 32
I20250111 09:01:29.979660 17699 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 32 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.979808 17699 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 29.10, Last appended by leader: 10, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.981305 17723 tablet_replica.cc:440] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c: TxnStatusTablet state changed. Reason: New leader b5fb421fa3354606aecbeafe55a0565c. Latest consensus state: current_term: 1 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
I20250111 09:01:29.981397 17723 tablet_replica.cc:443] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:29.981602 17725 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:01:29.981766 17725 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:01:29.982141 17180 catalog_manager.cc:5526] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 0 to 1, leader changed from <none> to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 1 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:29.983008 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 28 to 30, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 30 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.986083 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 30 to 32, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 32 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.989122 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:29.989586 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.989697 17467 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 30 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.989756 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 30 FOLLOWER]: Advancing to term 31
I20250111 09:01:29.990504 17467 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 31 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.990810 17467 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 31 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.991138 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 31 candidate_status { last_received { term: 28 index: 10 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.991197 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 31 candidate_status { last_received { term: 28 index: 10 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.991245 17339 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 30 LEADER]: Stepping down as leader of term 30
I20250111 09:01:29.991282 17339 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.991302 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 30 FOLLOWER]: Advancing to term 31
I20250111 09:01:29.991407 17339 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 30.10, Last appended by leader: 10, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.991523 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 30 FOLLOWER]: Advancing to term 31
I20250111 09:01:29.992113 17613 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 31.
I20250111 09:01:29.992358 17339 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 31 because replica has last-logged OpId of term: 30 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 10.
I20250111 09:01:29.992375 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:29.992522 17695 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 31 FOLLOWER]: Leader election won for term 31
I20250111 09:01:29.992588 17695 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 31 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.992704 17695 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 28.10, Last appended by leader: 10, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.993088 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:29.993196 17467 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 32 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:29.993245 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 32 FOLLOWER]: Advancing to term 33
I20250111 09:01:29.994062 17467 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 33 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.994379 17467 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 33 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:29.995067 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 33 candidate_status { last_received { term: 30 index: 11 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:29.995172 17339 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 32 LEADER]: Stepping down as leader of term 32
I20250111 09:01:29.995219 17339 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:29.995357 17339 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 32.11, Last appended by leader: 11, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.995424 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 33 candidate_status { last_received { term: 30 index: 11 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:29.995501 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 32 FOLLOWER]: Advancing to term 33
I20250111 09:01:29.995523 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 32 FOLLOWER]: Advancing to term 33
I20250111 09:01:29.995617 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 30 to 31, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 31 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:29.996244 17613 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 33.
I20250111 09:01:29.996302 17339 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 33 because replica has last-logged OpId of term: 32 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 11.
I20250111 09:01:29.996585 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:29.996850 17695 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 33 FOLLOWER]: Leader election won for term 33
I20250111 09:01:29.996938 17695 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 33 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:29.997080 17695 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 30.11, Last appended by leader: 11, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:29.998533 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 32 to 33, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 33 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:30.010406 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:30.010880 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:30.010985 17613 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 31 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.011041 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 31 FOLLOWER]: Advancing to term 32
I20250111 09:01:30.011744 17613 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 32 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.012061 17613 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 32 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:30.012374 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 32 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.012468 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 31 FOLLOWER]: Advancing to term 32
I20250111 09:01:30.012651 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 32 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.012679 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:30.012733 17467 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 31 LEADER]: Stepping down as leader of term 31
I20250111 09:01:30.012766 17467 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.012780 17613 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 33 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.012826 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 33 FOLLOWER]: Advancing to term 34
I20250111 09:01:30.012879 17467 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 31.11, Last appended by leader: 11, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.012977 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 31 FOLLOWER]: Advancing to term 32
I20250111 09:01:30.013226 17339 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 32 because replica has last-logged OpId of term: 30 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:30.013587 17613 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 34 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.013746 17467 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 32 because replica has last-logged OpId of term: 31 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:30.014051 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:30.014153 17613 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 34 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:30.014194 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 34 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.014366 17723 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20250111 09:01:30.014508 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 33 FOLLOWER]: Advancing to term 34
I20250111 09:01:30.014516 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 34 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.014595 17467 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 33 LEADER]: Stepping down as leader of term 33
I20250111 09:01:30.014642 17467 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.014767 17467 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 33.12, Last appended by leader: 12, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.014882 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 33 FOLLOWER]: Advancing to term 34
I20250111 09:01:30.015199 17339 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 34 because replica has last-logged OpId of term: 32 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:30.015581 17467 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 34 because replica has last-logged OpId of term: 33 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:30.015887 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:30.016026 17723 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20250111 09:01:30.016592 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:30.017091 17339 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:30.017206 17339 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 32 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.017266 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 32 FOLLOWER]: Advancing to term 33
I20250111 09:01:30.017773 17339 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 33 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.018085 17339 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 33 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.018410 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 33 candidate_status { last_received { term: 30 index: 10 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.018549 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 32 FOLLOWER]: Advancing to term 33
I20250111 09:01:30.018528 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 33 candidate_status { last_received { term: 30 index: 10 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.018615 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 32 FOLLOWER]: Advancing to term 33
I20250111 09:01:30.018898 17339 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:30.019002 17339 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 34 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.019058 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 34 FOLLOWER]: Advancing to term 35
I20250111 09:01:30.019317 17613 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 33.
I20250111 09:01:30.019546 17467 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 33 because replica has last-logged OpId of term: 31 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 10.
I20250111 09:01:30.019594 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:30.019744 17687 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 33 FOLLOWER]: Leader election won for term 33
I20250111 09:01:30.019726 17339 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 35 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.019834 17687 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 33 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:30.019944 17687 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 30.10, Last appended by leader: 10, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.020022 17339 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 35 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.021037 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 35 candidate_status { last_received { term: 32 index: 11 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.021171 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 34 FOLLOWER]: Advancing to term 35
I20250111 09:01:30.021184 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 35 candidate_status { last_received { term: 32 index: 11 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.021270 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 34 FOLLOWER]: Advancing to term 35
I20250111 09:01:30.021306 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 31 to 33, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 33 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:30.022018 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 35.
I20250111 09:01:30.022277 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:30.022428 17699 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 35 FOLLOWER]: Leader election won for term 35
I20250111 09:01:30.022502 17699 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 35 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:30.022605 17699 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 32.11, Last appended by leader: 11, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.023072 17467 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 35 because replica has last-logged OpId of term: 33 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 32 index: 11.
I20250111 09:01:30.024199 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 33 to 35, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 35 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:30.032222 17339 raft_consensus.cc:1270] T 2f60a4d7104242869ce22b7a9c250295 P fb270e42d7c9400f8c05bb9565db1972 [term 1 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:30.032222 17467 raft_consensus.cc:1270] T 2f60a4d7104242869ce22b7a9c250295 P 262623ac6df745c785ce267ea39bd265 [term 1 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:01:30.032593 17690 consensus_queue.cc:1035] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:30.032717 17723 consensus_queue.cc:1035] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:30.035461 17691 tablet_replica.cc:440] T 2f60a4d7104242869ce22b7a9c250295 P 262623ac6df745c785ce267ea39bd265: TxnStatusTablet state changed. Reason: New leader b5fb421fa3354606aecbeafe55a0565c. Latest consensus state: current_term: 1 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
I20250111 09:01:30.035550 17691 tablet_replica.cc:443] T 2f60a4d7104242869ce22b7a9c250295 P 262623ac6df745c785ce267ea39bd265: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:30.035530 17687 tablet_replica.cc:440] T 2f60a4d7104242869ce22b7a9c250295 P fb270e42d7c9400f8c05bb9565db1972: TxnStatusTablet state changed. Reason: New leader b5fb421fa3354606aecbeafe55a0565c. Latest consensus state: current_term: 1 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
I20250111 09:01:30.035610 17687 tablet_replica.cc:443] T 2f60a4d7104242869ce22b7a9c250295 P fb270e42d7c9400f8c05bb9565db1972: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:30.037380 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:30.037791 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:30.037886 17476 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 33 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.037941 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 33 FOLLOWER]: Advancing to term 34
I20250111 09:01:30.038637 17476 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 34 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.038933 17476 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 34 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.039063 17690 tablet_replica.cc:440] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
I20250111 09:01:30.039126 17690 tablet_replica.cc:443] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:30.039502 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 34 candidate_status { last_received { term: 31 index: 11 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.039506 17691 tablet_replica.cc:440] T 2f60a4d7104242869ce22b7a9c250295 P 262623ac6df745c785ce267ea39bd265: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
I20250111 09:01:30.039579 17691 tablet_replica.cc:443] T 2f60a4d7104242869ce22b7a9c250295 P 262623ac6df745c785ce267ea39bd265: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:30.039577 17338 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 33 LEADER]: Stepping down as leader of term 33
I20250111 09:01:30.039610 17338 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:30.039706 17338 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 33.11, Last appended by leader: 11, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.039831 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 33 FOLLOWER]: Advancing to term 34
I20250111 09:01:30.040053 17729 mvcc.cc:204] Tried to move back new op lower bound from 7113056624767434752 to 7113056624558551040. Current Snapshot: MvccSnapshot[applied={T|T < 7113056624767434752}]
I20250111 09:01:30.040196 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 34 candidate_status { last_received { term: 31 index: 11 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.040280 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 33 FOLLOWER]: Advancing to term 34
I20250111 09:01:30.040560 17338 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 34 because replica has last-logged OpId of term: 33 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 11.
I20250111 09:01:30.040941 17613 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 34.
I20250111 09:01:30.041316 17728 mvcc.cc:204] Tried to move back new op lower bound from 7113056624767434752 to 7113056624558551040. Current Snapshot: MvccSnapshot[applied={T|T < 7113056624767434752}]
I20250111 09:01:30.041503 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:30.041682 17691 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 34 FOLLOWER]: Leader election won for term 34
I20250111 09:01:30.041754 17691 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 34 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.041855 17691 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 31.11, Last appended by leader: 11, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.043771 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:30.043881 17467 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 35 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.043932 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 35 FOLLOWER]: Advancing to term 36
I20250111 09:01:30.044644 17467 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 36 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.044962 17467 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 36 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.045622 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 36 candidate_status { last_received { term: 33 index: 12 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.045712 17338 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 35 LEADER]: Stepping down as leader of term 35
I20250111 09:01:30.045746 17338 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:30.045887 17338 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 35.12, Last appended by leader: 12, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.045984 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 35 FOLLOWER]: Advancing to term 36
I20250111 09:01:30.046648 17338 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 36 because replica has last-logged OpId of term: 35 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 33 index: 12.
I20250111 09:01:30.047447 17723 tablet_replica.cc:440] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
I20250111 09:01:30.047526 17723 tablet_replica.cc:443] T 2f60a4d7104242869ce22b7a9c250295 P b5fb421fa3354606aecbeafe55a0565c: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:01:30.047562 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 33 to 34, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 34 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:30.049597 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 36 candidate_status { last_received { term: 33 index: 12 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.049680 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 35 FOLLOWER]: Advancing to term 36
I20250111 09:01:30.050305 17613 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 36.
I20250111 09:01:30.050563 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:30.050729 17695 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 36 FOLLOWER]: Leader election won for term 36
I20250111 09:01:30.050812 17695 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 36 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.050911 17695 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 33.12, Last appended by leader: 12, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.051627 17730 mvcc.cc:204] Tried to move back new op lower bound from 7113056624767434752 to 7113056624558551040. Current Snapshot: MvccSnapshot[applied={T|T < 7113056624767434752}]
I20250111 09:01:30.052120 17687 tablet_replica.cc:440] T 2f60a4d7104242869ce22b7a9c250295 P fb270e42d7c9400f8c05bb9565db1972: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
I20250111 09:01:30.052193 17687 tablet_replica.cc:443] T 2f60a4d7104242869ce22b7a9c250295 P fb270e42d7c9400f8c05bb9565db1972: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:01:30.052760 17180 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 35 to 36, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 36 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:30.063207 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:30.063673 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:30.063767 17613 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 34 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.063805 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 34 FOLLOWER]: Advancing to term 35
I20250111 09:01:30.064445 17613 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 35 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.064774 17613 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 35 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:30.065021 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 35 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.065114 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 34 FOLLOWER]: Advancing to term 35
I20250111 09:01:30.065101 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 35 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.065203 17467 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 34 LEADER]: Stepping down as leader of term 34
I20250111 09:01:30.065239 17467 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.065343 17467 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 34.12, Last appended by leader: 12, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.065443 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 34 FOLLOWER]: Advancing to term 35
I20250111 09:01:30.065907 17339 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 35 because replica has last-logged OpId of term: 33 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:30.066171 17467 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 35 because replica has last-logged OpId of term: 34 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:30.066803 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:30.066960 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:30.067004 17690 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20250111 09:01:30.067065 17613 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 36 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.067126 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 36 FOLLOWER]: Advancing to term 37
I20250111 09:01:30.067785 17613 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 37 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.068037 17613 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 37 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:30.068221 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 37 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.068307 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 36 FOLLOWER]: Advancing to term 37
I20250111 09:01:30.068320 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 37 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.068408 17467 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 36 LEADER]: Stepping down as leader of term 36
I20250111 09:01:30.068439 17467 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.068531 17467 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 36.13, Last appended by leader: 13, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.068621 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 36 FOLLOWER]: Advancing to term 37
I20250111 09:01:30.068991 17339 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 37 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 37 because replica has last-logged OpId of term: 35 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:30.069293 17467 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 37 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 37 because replica has last-logged OpId of term: 36 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:30.069628 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:30.069800 17690 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
I20250111 09:01:30.070394 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:30.070905 17339 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:30.070976 17339 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 35 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.071015 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 35 FOLLOWER]: Advancing to term 36
I20250111 09:01:30.071525 17339 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 36 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.071771 17339 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 36 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.072307 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 36 candidate_status { last_received { term: 33 index: 11 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.072391 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 35 FOLLOWER]: Advancing to term 36
I20250111 09:01:30.072625 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 36 candidate_status { last_received { term: 33 index: 11 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.072695 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 35 FOLLOWER]: Advancing to term 36
I20250111 09:01:30.073071 17476 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 36 because replica has last-logged OpId of term: 34 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 33 index: 11.
I20250111 09:01:30.073197 17613 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 36.
I20250111 09:01:30.073637 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:01:30.073784 17687 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 36 FOLLOWER]: Leader election won for term 36
I20250111 09:01:30.073858 17687 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 36 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:30.073925 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:30.074002 17338 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 37 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.073967 17687 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 33.11, Last appended by leader: 11, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.074056 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 37 FOLLOWER]: Advancing to term 38
I20250111 09:01:30.074777 17338 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 38 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.075059 17338 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 38 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.075824 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 38 candidate_status { last_received { term: 35 index: 12 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.075906 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 37 FOLLOWER]: Advancing to term 38
I20250111 09:01:30.075989 17180 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 34 to 36, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 36 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:30.076932 17613 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 38.
I20250111 09:01:30.077171 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:30.077291 17699 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 38 FOLLOWER]: Leader election won for term 38
I20250111 09:01:30.077347 17699 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 38 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:30.077433 17699 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 35.12, Last appended by leader: 12, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.077495 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 38 candidate_status { last_received { term: 35 index: 12 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.077560 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 37 FOLLOWER]: Advancing to term 38
I20250111 09:01:30.078182 17467 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 38 because replica has last-logged OpId of term: 36 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 35 index: 12.
I20250111 09:01:30.079541 17180 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 36 to 38, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 38 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:30.080955 17467 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 38 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 36 index: 13. Preceding OpId from leader: term: 38 index: 14. (index mismatch)
I20250111 09:01:30.081190 17613 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 38 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 38 index: 14. (index mismatch)
I20250111 09:01:30.081562 17687 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:30.081672 17699 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:30.081764 17699 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer 262623ac6df745c785ce267ea39bd265 log is divergent from this leader: its last log entry 36.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20250111 09:01:30.082412 17467 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 0
I20250111 09:01:30.082479 17467 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20250111 09:01:30.082523 17467 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 38 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.082573 17467 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20250111 09:01:30.082592 17467 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 38 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.082617 17467 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20250111 09:01:30.082636 17467 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 38 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.082661 17467 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted NO_OP operation due to leader change: 9.4
I20250111 09:01:30.082679 17467 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 38 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.082705 17467 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted NO_OP operation due to leader change: 12.5
I20250111 09:01:30.082721 17467 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 38 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.082744 17467 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted NO_OP operation due to leader change: 15.6
I20250111 09:01:30.082772 17467 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 38 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.082795 17467 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted NO_OP operation due to leader change: 18.7
I20250111 09:01:30.082811 17467 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 38 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.082834 17467 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted NO_OP operation due to leader change: 21.8
I20250111 09:01:30.082851 17467 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 38 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.082875 17467 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted NO_OP operation due to leader change: 24.9
I20250111 09:01:30.082891 17467 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 38 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.082916 17467 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted NO_OP operation due to leader change: 27.10
I20250111 09:01:30.082935 17467 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 38 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.082963 17467 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted NO_OP operation due to leader change: 30.11
I20250111 09:01:30.082981 17467 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 38 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.083004 17467 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted NO_OP operation due to leader change: 33.12
I20250111 09:01:30.083021 17467 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 38 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.083045 17467 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted NO_OP operation due to leader change: 36.13
I20250111 09:01:30.083062 17467 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 38 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.102723 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:30.104038 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:30.104138 17476 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 36 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.104185 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 36 FOLLOWER]: Advancing to term 37
I20250111 09:01:30.104966 17476 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 37 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.105296 17476 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 37 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.106089 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 37 candidate_status { last_received { term: 34 index: 12 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.106179 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 36 FOLLOWER]: Advancing to term 37
I20250111 09:01:30.106331 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:30.106432 17467 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 38 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.106482 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 38 FOLLOWER]: Advancing to term 39
I20250111 09:01:30.106936 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 37.
I20250111 09:01:30.107051 17613 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 37 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 36. Current term is 37. Ops: []
I20250111 09:01:30.107141 17467 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 39 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.107416 17467 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 39 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.107419 17745 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:30.107854 17687 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 36 LEADER]: Stepping down as leader of term 36
I20250111 09:01:30.107903 17687 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 36 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:30.108039 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 39 candidate_status { last_received { term: 38 index: 17 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.108026 17687 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 36.13, Last appended by leader: 13, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.108121 17339 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 38 LEADER]: Stepping down as leader of term 38
I20250111 09:01:30.108126 17687 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 36 FOLLOWER]: Advancing to term 37
I20250111 09:01:30.108160 17339 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 38 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:30.108989 17339 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 38.17, Last appended by leader: 17, Current term: 38, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.108757 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:30.109138 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 38 FOLLOWER]: Advancing to term 39
I20250111 09:01:30.109287 17742 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 37 FOLLOWER]: Leader election won for term 37
I20250111 09:01:30.109309 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 37 candidate_status { last_received { term: 34 index: 12 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.109421 17338 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 37 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 37 because replica has last-logged OpId of term: 36 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 34 index: 12.
I20250111 09:01:30.109720 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 39 candidate_status { last_received { term: 38 index: 17 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.109812 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 38 FOLLOWER]: Advancing to term 39
I20250111 09:01:30.109844 17339 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 39.
I20250111 09:01:30.109920 17742 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 37 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.110071 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:30.110215 17691 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 39 FOLLOWER]: Leader election won for term 39
I20250111 09:01:30.110268 17691 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 39 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.110360 17691 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 38.17, Last appended by leader: 17, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.110468 17613 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 39.
I20250111 09:01:30.110071 17742 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 34.12, Last appended by leader: 12, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.111330 17476 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 37 LEADER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 36. Current term is 37. Ops: []
I20250111 09:01:30.112025 17180 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 36 to 37, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 37 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:30.114461 17180 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 38 to 39, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 39 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:30.130163 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:30.130597 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:30.130699 17613 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 37 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.130750 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 37 FOLLOWER]: Advancing to term 38
I20250111 09:01:30.131525 17613 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 38 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.131839 17613 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 38 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:30.132295 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 38 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.132298 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 38 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.132390 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 37 FOLLOWER]: Advancing to term 38
I20250111 09:01:30.132393 17467 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 37 LEADER]: Stepping down as leader of term 37
I20250111 09:01:30.132429 17467 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 37 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.132534 17467 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 37.13, Last appended by leader: 13, Current term: 37, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.132645 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 37 FOLLOWER]: Advancing to term 38
I20250111 09:01:30.133159 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:30.133250 17613 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 39 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.133301 17338 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 38 because replica has last-logged OpId of term: 36 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:30.133304 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 39 FOLLOWER]: Advancing to term 40
I20250111 09:01:30.133445 17467 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 38 because replica has last-logged OpId of term: 37 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:01:30.133808 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 38 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:30.134054 17613 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 40 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.134364 17613 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 40 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:30.134786 17690 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 38 FOLLOWER]: Leader election lost for term 38. Reason: could not achieve majority
I20250111 09:01:30.134891 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 40 candidate_status { last_received { term: 38 index: 17 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.135030 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 39 FOLLOWER]: Advancing to term 40
I20250111 09:01:30.136682 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:30.137403 17339 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 40.
I20250111 09:01:30.137811 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 40 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
W20250111 09:01:30.137943 17738 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations
W20250111 09:01:30.138037 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.138046 17690 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 40 FOLLOWER]: Leader election won for term 40
I20250111 09:01:30.138123 17690 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 40 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:30.138221 17690 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 38.17, Last appended by leader: 17, Current term: 40, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.139318 17180 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 39 to 40, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 40 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:30.141577 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:30.141664 17338 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 38 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.141719 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 38 FOLLOWER]: Advancing to term 39
I20250111 09:01:30.142351 17338 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 39 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.142619 17338 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 39 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.142944 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 39 candidate_status { last_received { term: 36 index: 13 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.143075 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 38 FOLLOWER]: Advancing to term 39
W20250111 09:01:30.143632 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.144021 17613 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 39.
I20250111 09:01:30.142944 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 39 candidate_status { last_received { term: 36 index: 13 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.144709 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 38 FOLLOWER]: Advancing to term 39
I20250111 09:01:30.144948 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:30.145457 17745 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 39 FOLLOWER]: Leader election won for term 39
I20250111 09:01:30.145535 17467 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 39 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 39 because replica has last-logged OpId of term: 37 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 36 index: 13.
I20250111 09:01:30.134716 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 40 candidate_status { last_received { term: 38 index: 17 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.145820 17476 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 39 LEADER]: Stepping down as leader of term 39
I20250111 09:01:30.145852 17476 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 39 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.145956 17476 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 39.18, Last appended by leader: 18, Current term: 39, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.146059 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 39 FOLLOWER]: Advancing to term 40
I20250111 09:01:30.146135 17745 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 39 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:30.146389 17745 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 36.13, Last appended by leader: 13, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.146755 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 40 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 40 because replica has last-logged OpId of term: 39 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 38 index: 17.
I20250111 09:01:30.147508 17339 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:30.147622 17339 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 40 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.147677 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 40 FOLLOWER]: Advancing to term 41
I20250111 09:01:30.147680 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 37 to 39, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 39 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:30.148336 17339 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 41 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.148617 17339 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 41 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.150867 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 41 candidate_status { last_received { term: 38 index: 17 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.150887 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 41 candidate_status { last_received { term: 38 index: 17 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.150951 17613 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 40 LEADER]: Stepping down as leader of term 40
I20250111 09:01:30.150964 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 40 FOLLOWER]: Advancing to term 41
I20250111 09:01:30.150990 17613 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 40 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:30.151121 17613 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 40.18, Last appended by leader: 18, Current term: 40, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.151288 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 40 FOLLOWER]: Advancing to term 41
I20250111 09:01:30.151667 17477 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 41 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 41 because replica has last-logged OpId of term: 39 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 38 index: 17.
I20250111 09:01:30.151823 17613 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 41 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 41 because replica has last-logged OpId of term: 40 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 38 index: 17.
I20250111 09:01:30.152026 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 41 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:30.152202 17745 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 41 FOLLOWER]: Leader election lost for term 41. Reason: could not achieve majority
I20250111 09:01:30.157181 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:30.157603 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:30.157707 17477 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 39 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.157759 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 39 FOLLOWER]: Advancing to term 40
I20250111 09:01:30.158455 17477 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 40 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.158736 17477 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 40 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.159111 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 40 candidate_status { last_received { term: 37 index: 13 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.159206 17338 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 39 LEADER]: Stepping down as leader of term 39
I20250111 09:01:30.159250 17338 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 39 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:30.159349 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:30.159353 17338 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 39.14, Last appended by leader: 14, Current term: 39, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.159431 17477 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 41 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.159472 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 41 FOLLOWER]: Advancing to term 42
I20250111 09:01:30.159478 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 39 FOLLOWER]: Advancing to term 40
I20250111 09:01:30.160034 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 40 candidate_status { last_received { term: 37 index: 13 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.160140 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 39 FOLLOWER]: Advancing to term 40
I20250111 09:01:30.160173 17477 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 42 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.160219 17338 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 40 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 40 because replica has last-logged OpId of term: 39 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 37 index: 13.
I20250111 09:01:30.160421 17477 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 42 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.160756 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 42 candidate_status { last_received { term: 39 index: 18 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.160818 17613 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 40.
I20250111 09:01:30.160849 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 41 FOLLOWER]: Advancing to term 42
I20250111 09:01:30.161501 17338 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 42 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 42.
I20250111 09:01:30.161827 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 42 candidate_status { last_received { term: 39 index: 18 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.161855 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 40 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:30.161908 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 41 FOLLOWER]: Advancing to term 42
I20250111 09:01:30.162055 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 42 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:30.162189 17742 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 40 FOLLOWER]: Leader election won for term 40
I20250111 09:01:30.162254 17742 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 40 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.162348 17742 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 37.13, Last appended by leader: 13, Current term: 40, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.162523 17613 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 42 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 42 because replica has last-logged OpId of term: 40 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 39 index: 18.
I20250111 09:01:30.162734 17742 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 42 FOLLOWER]: Leader election won for term 42
I20250111 09:01:30.162827 17742 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 42 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.162915 17742 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 39.18, Last appended by leader: 18, Current term: 42, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.164258 17180 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 39 to 40, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 40 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:30.164450 17180 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 40 to 42, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 42 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:30.168771 17613 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 42 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 40 index: 18. Preceding OpId from leader: term: 42 index: 20. (index mismatch)
I20250111 09:01:30.168936 17614 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 40 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 40 index: 15. (index mismatch)
I20250111 09:01:30.169374 17339 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 42 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 38 index: 17. Preceding OpId from leader: term: 42 index: 20. (index mismatch)
I20250111 09:01:30.169341 17695 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20250111 09:01:30.169437 17695 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 40.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20250111 09:01:30.169467 17340 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 40 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 39 index: 14. Preceding OpId from leader: term: 40 index: 15. (index mismatch)
I20250111 09:01:30.169708 17695 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:30.170006 17614 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 17
I20250111 09:01:30.170061 17614 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted NO_OP operation due to leader change: 40.18
I20250111 09:01:30.170095 17614 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 42 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.171064 17744 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20250111 09:01:30.171286 17744 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:30.171339 17744 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer fb270e42d7c9400f8c05bb9565db1972 log is divergent from this leader: its last log entry 39.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20250111 09:01:30.171921 17339 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 0
I20250111 09:01:30.171977 17339 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20250111 09:01:30.172008 17339 raft_consensus.cc:2883] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 40 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.172040 17339 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted NO_OP operation due to leader change: 6.2
I20250111 09:01:30.172057 17339 raft_consensus.cc:2883] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 40 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.172080 17339 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted NO_OP operation due to leader change: 9.3
I20250111 09:01:30.172096 17339 raft_consensus.cc:2883] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 40 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.172117 17339 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted NO_OP operation due to leader change: 12.4
I20250111 09:01:30.172133 17339 raft_consensus.cc:2883] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 40 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.172154 17339 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted NO_OP operation due to leader change: 15.5
I20250111 09:01:30.172170 17339 raft_consensus.cc:2883] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 40 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.172191 17339 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted NO_OP operation due to leader change: 18.6
I20250111 09:01:30.172207 17339 raft_consensus.cc:2883] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 40 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.172232 17339 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted NO_OP operation due to leader change: 21.7
I20250111 09:01:30.172248 17339 raft_consensus.cc:2883] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 40 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.172269 17339 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted NO_OP operation due to leader change: 24.8
I20250111 09:01:30.172286 17339 raft_consensus.cc:2883] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 40 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.172307 17339 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted NO_OP operation due to leader change: 27.9
I20250111 09:01:30.172322 17339 raft_consensus.cc:2883] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 40 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.172343 17339 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted NO_OP operation due to leader change: 30.10
I20250111 09:01:30.172358 17339 raft_consensus.cc:2883] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 40 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.172382 17339 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted NO_OP operation due to leader change: 33.11
I20250111 09:01:30.172398 17339 raft_consensus.cc:2883] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 40 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.172420 17339 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted NO_OP operation due to leader change: 36.12
I20250111 09:01:30.172434 17339 raft_consensus.cc:2883] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 40 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.172456 17339 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 36.13
W20250111 09:01:30.172552 17339 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Aborted: Op aborted by new leader
I20250111 09:01:30.172732 17339 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted NO_OP operation due to leader change: 39.14
I20250111 09:01:30.172767 17339 raft_consensus.cc:2883] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 40 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.174985 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:30.176261 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:30.176358 17613 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 40 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.176399 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 40 FOLLOWER]: Advancing to term 41
I20250111 09:01:30.176661 17730 mvcc.cc:204] Tried to move back new op lower bound from 7113056625327304704 to 7113056625308250112. Current Snapshot: MvccSnapshot[applied={T|T < 7113056625327304704}]
I20250111 09:01:30.177085 17613 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 41 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.177384 17613 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 41 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:30.178480 17729 mvcc.cc:204] Tried to move back new op lower bound from 7113056625328914432 to 7113056624239689728. Current Snapshot: MvccSnapshot[applied={T|T < 7113056625328914432}]
I20250111 09:01:30.178669 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 41 candidate_status { last_received { term: 40 index: 15 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.178747 17477 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 40 LEADER]: Stepping down as leader of term 40
I20250111 09:01:30.178788 17477 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 40 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.178869 17477 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 40.15, Last appended by leader: 15, Current term: 40, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.178954 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 40 FOLLOWER]: Advancing to term 41
I20250111 09:01:30.179672 17477 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 41 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 41.
I20250111 09:01:30.180039 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:30.180130 17613 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 42 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.180172 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 42 FOLLOWER]: Advancing to term 43
I20250111 09:01:30.180799 17613 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 43 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.181100 17613 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 43 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:30.181347 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 43 candidate_status { last_received { term: 42 index: 21 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.181381 17614 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 41 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 40. Current term is 41. Ops: []
W20250111 09:01:30.181391 17729 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: Replica 262623ac6df745c785ce267ea39bd265 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 41 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
I20250111 09:01:30.181582 17612 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 43 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 42. Current term is 43. Ops: []
I20250111 09:01:30.181991 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 41 candidate_status { last_received { term: 40 index: 15 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.182065 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 41 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:30.182075 17744 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: INVALID_TERM, Last received: 42.21, Next index: 22, Last known committed idx: 20, Time since last communication: 0.000s
I20250111 09:01:30.182195 17744 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 42 LEADER]: Stepping down as leader of term 42
I20250111 09:01:30.182230 17744 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 42 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.182337 17744 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 42.21, Last appended by leader: 21, Current term: 42, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.182447 17744 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 42 FOLLOWER]: Advancing to term 43
I20250111 09:01:30.182446 17690 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 41 FOLLOWER]: Leader election won for term 41
I20250111 09:01:30.182511 17690 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 41 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:30.182627 17690 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 40.15, Last appended by leader: 15, Current term: 41, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:30.183079 17749 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations
W20250111 09:01:30.183168 17749 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.184639 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.184993 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 43 candidate_status { last_received { term: 42 index: 21 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.185843 17477 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 43 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 43.
I20250111 09:01:30.185886 17180 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 40 to 41, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 41 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:30.186451 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 43 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:30.186620 17747 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 43 FOLLOWER]: Leader election won for term 43
I20250111 09:01:30.186691 17747 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 43 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:30.186798 17747 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 42.21, Last appended by leader: 21, Current term: 43, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.189332 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 42 to 43, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 43 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
W20250111 09:01:30.202062 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.216426 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:30.216830 17339 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:30.216922 17339 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 40 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.216976 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 40 FOLLOWER]: Advancing to term 41
I20250111 09:01:30.217736 17339 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 41 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.218019 17339 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 41 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.218258 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 41 candidate_status { last_received { term: 40 index: 15 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.218369 17477 raft_consensus.cc:2388] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 41 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 in current term 41: Already voted for candidate b5fb421fa3354606aecbeafe55a0565c in this term.
I20250111 09:01:30.218385 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 41 candidate_status { last_received { term: 40 index: 15 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.218502 17612 raft_consensus.cc:2388] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 41 LEADER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 in current term 41: Already voted for candidate b5fb421fa3354606aecbeafe55a0565c in this term.
I20250111 09:01:30.218640 17339 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:30.218681 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 41 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:30.218729 17339 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 42 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.218827 17760 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 41 FOLLOWER]: Leader election lost for term 41. Reason: could not achieve majority
I20250111 09:01:30.218886 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 42 FOLLOWER]: Advancing to term 43
I20250111 09:01:30.219450 17339 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 43 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.219765 17339 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 43 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.220028 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 43 candidate_status { last_received { term: 42 index: 21 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.220031 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 43 candidate_status { last_received { term: 42 index: 21 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.220130 17477 raft_consensus.cc:2388] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 43 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 in current term 43: Already voted for candidate b5fb421fa3354606aecbeafe55a0565c in this term.
I20250111 09:01:30.220132 17612 raft_consensus.cc:2388] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 43 LEADER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 in current term 43: Already voted for candidate b5fb421fa3354606aecbeafe55a0565c in this term.
I20250111 09:01:30.220381 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 43 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:30.220528 17760 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 43 FOLLOWER]: Leader election lost for term 43. Reason: could not achieve majority
I20250111 09:01:30.224292 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:30.224741 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:30.224850 17477 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 41 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.224910 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 41 FOLLOWER]: Advancing to term 42
I20250111 09:01:30.225631 17477 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 42 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.225901 17477 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 42 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.226151 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 42 candidate_status { last_received { term: 40 index: 15 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.226240 17612 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 41 LEADER]: Stepping down as leader of term 41
I20250111 09:01:30.226275 17612 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 41 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:30.226305 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 42 candidate_status { last_received { term: 40 index: 15 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.226387 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:30.226387 17612 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 41.16, Last appended by leader: 16, Current term: 41, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.226433 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 41 FOLLOWER]: Advancing to term 42
I20250111 09:01:30.226460 17477 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 43 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.226506 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 43 FOLLOWER]: Advancing to term 44
I20250111 09:01:30.226501 17612 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 41 FOLLOWER]: Advancing to term 42
I20250111 09:01:30.226938 17339 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 42 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 42.
I20250111 09:01:30.227159 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 42 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:30.227152 17477 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 44 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.227258 17612 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 42 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 42 because replica has last-logged OpId of term: 41 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 40 index: 15.
I20250111 09:01:30.227309 17744 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 42 FOLLOWER]: Leader election won for term 42
I20250111 09:01:30.227382 17744 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 42 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.227475 17744 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 40.15, Last appended by leader: 15, Current term: 42, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.227676 17477 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 44 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.227970 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 44 candidate_status { last_received { term: 42 index: 21 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.228056 17612 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 43 LEADER]: Stepping down as leader of term 43
I20250111 09:01:30.228096 17612 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 43 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:30.228207 17612 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 43.22, Last appended by leader: 22, Current term: 43, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.228320 17612 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 43 FOLLOWER]: Advancing to term 44
I20250111 09:01:30.228542 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 41 to 42, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 42 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:30.229064 17612 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 44 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 44 because replica has last-logged OpId of term: 43 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 42 index: 21.
I20250111 09:01:30.229326 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 44 candidate_status { last_received { term: 42 index: 21 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.229410 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 43 FOLLOWER]: Advancing to term 44
I20250111 09:01:30.230046 17339 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 44 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 44.
I20250111 09:01:30.230242 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 44 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters: b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:30.230388 17744 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 44 FOLLOWER]: Leader election won for term 44
I20250111 09:01:30.230486 17744 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 44 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.230584 17744 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 42.21, Last appended by leader: 21, Current term: 44, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.231832 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 43 to 44, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 44 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:30.238667 17761 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations
W20250111 09:01:30.238762 17761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.244647 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.251672 17612 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 42 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 41 index: 16. Preceding OpId from leader: term: 42 index: 17. (index mismatch)
I20250111 09:01:30.251993 17691 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:01:30.252084 17691 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 41.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20250111 09:01:30.252357 17339 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 42 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 40 index: 15. Preceding OpId from leader: term: 42 index: 17. (index mismatch)
I20250111 09:01:30.252746 17691 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20250111 09:01:30.252965 17612 raft_consensus.cc:1212] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 42 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[2.1-42.17] Dedup: 40.15->[42.16-42.17]
I20250111 09:01:30.253073 17612 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 15
I20250111 09:01:30.253095 17612 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted NO_OP operation due to leader change: 41.16
I20250111 09:01:30.253118 17612 raft_consensus.cc:2883] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 42 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20250111 09:01:30.256511 17728 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.256825 17728 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.002s ago.
W20250111 09:01:30.257109 17728 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.002s ago.
W20250111 09:01:30.257372 17728 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.002s ago.
W20250111 09:01:30.257616 17728 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.002s ago.
I20250111 09:01:30.257812 17728 mvcc.cc:204] Tried to move back new op lower bound from 7113056625328914432 to 7113056624239689728. Current Snapshot: MvccSnapshot[applied={T|T < 7113056625328914432}]
I20250111 09:01:30.257859 17614 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 44 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 43 index: 22. Preceding OpId from leader: term: 44 index: 23. (index mismatch)
I20250111 09:01:30.258143 17339 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 44 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 42 index: 21. Preceding OpId from leader: term: 44 index: 23. (index mismatch)
I20250111 09:01:30.258185 17691 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.000s
I20250111 09:01:30.258248 17691 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 43.22 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 20
I20250111 09:01:30.258961 17614 raft_consensus.cc:1212] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 44 FOLLOWER]: Deduplicated request from leader. Original: 42.20->[42.21-44.23] Dedup: 42.21->[44.22-44.23]
I20250111 09:01:30.259029 17614 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 21
I20250111 09:01:30.259058 17614 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted NO_OP operation due to leader change: 43.22
I20250111 09:01:30.259081 17614 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 44 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.259225 17743 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.000s
W20250111 09:01:30.264689 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.265226 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.268193 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.278378 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:30.278770 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:30.278863 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 42 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.278909 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 42 FOLLOWER]: Advancing to term 43
I20250111 09:01:30.279771 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 43 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.280056 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 43 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:30.280257 17613 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 43 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 42. Current term is 43. Ops: [42.22-42.23]
I20250111 09:01:30.280689 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:30.280776 17613 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 44 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.280827 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 44 FOLLOWER]: Advancing to term 45
I20250111 09:01:30.280860 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 43 candidate_status { last_received { term: 42 index: 21 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.280941 17477 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 42 LEADER]: Stepping down as leader of term 42
I20250111 09:01:30.280973 17477 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 42 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.281100 17477 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 42.23, Last appended by leader: 23, Current term: 42, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:30.281400 17749 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.281522 17613 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 45 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.281795 17613 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 45 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:30.282037 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 45 candidate_status { last_received { term: 44 index: 27 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.282878 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 45 candidate_status { last_received { term: 44 index: 27 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.282965 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 44 FOLLOWER]: Advancing to term 45
I20250111 09:01:30.283116 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 43 candidate_status { last_received { term: 42 index: 21 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.283223 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 42 FOLLOWER]: Advancing to term 43
I20250111 09:01:30.283794 17339 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 45 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 45.
I20250111 09:01:30.284083 17338 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 43 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 43.
I20250111 09:01:30.284406 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:30.284824 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:30.284922 17338 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 43 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.284969 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 43 FOLLOWER]: Advancing to term 44
W20250111 09:01:30.284890 17691 consensus_queue.cc:1159] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b5fb421fa3354606aecbeafe55a0565c" responder_term: 43 status { last_received { term: 42 index: 21 } last_committed_idx: 21 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 42. Current term is 43. Ops: [42.22-42.23]" } } last_received_current_leader { term: 0 index: 0 } }
I20250111 09:01:30.285351 17613 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 45 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 44. Current term is 45. Ops: [44.28-44.29]
I20250111 09:01:30.285696 17338 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 44 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.285897 17691 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: INVALID_TERM, Last received: 44.27, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20250111 09:01:30.286007 17691 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 44 LEADER]: Stepping down as leader of term 44
I20250111 09:01:30.286053 17691 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 44 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.286149 17691 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 44.29, Last appended by leader: 29, Current term: 44, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.286283 17691 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 44 FOLLOWER]: Advancing to term 45
I20250111 09:01:30.286319 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 45 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:01:30.286512 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 43 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
W20250111 09:01:30.286856 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.287472 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 42 FOLLOWER]: Advancing to term 43
I20250111 09:01:30.287560 17338 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 44 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.287806 17690 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 45 FOLLOWER]: Leader election won for term 45
I20250111 09:01:30.287878 17690 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 45 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:30.287981 17690 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 44.27, Last appended by leader: 27, Current term: 45, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.288226 17339 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:30.288293 17477 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 43 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 43 because replica has last-logged OpId of term: 42 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 42 index: 21.
I20250111 09:01:30.288311 17339 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 45 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.288358 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 45 FOLLOWER]: Advancing to term 46
I20250111 09:01:30.288394 17690 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 43 FOLLOWER]: Leader election won for term 43
I20250111 09:01:30.288452 17690 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 43 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:30.288540 17690 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 42.21, Last appended by leader: 21, Current term: 43, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:30.288632 17729 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: Replica 262623ac6df745c785ce267ea39bd265 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 43 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
I20250111 09:01:30.289115 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 44 candidate_status { last_received { term: 42 index: 21 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.289191 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 43 FOLLOWER]: Advancing to term 44
I20250111 09:01:30.289170 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 44 candidate_status { last_received { term: 42 index: 21 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.289242 17613 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 43 LEADER]: Stepping down as leader of term 43
I20250111 09:01:30.289276 17613 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 43 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:30.289357 17339 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 46 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.289379 17613 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 43.22, Last appended by leader: 22, Current term: 43, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.289467 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 43 FOLLOWER]: Advancing to term 44
I20250111 09:01:30.289641 17339 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 46 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.289891 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 46 candidate_status { last_received { term: 44 index: 26 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.289996 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 45 FOLLOWER]: Advancing to term 46
I20250111 09:01:30.290239 17613 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 44 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 44 because replica has last-logged OpId of term: 43 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 42 index: 21.
I20250111 09:01:30.290789 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 46 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 46 because replica has last-logged OpId of term: 44 index: 29, which is greater than that of the candidate, which has last-logged OpId of term: 44 index: 26.
I20250111 09:01:30.291260 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 46 candidate_status { last_received { term: 44 index: 26 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.291354 17614 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 45 LEADER]: Stepping down as leader of term 45
I20250111 09:01:30.291401 17614 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 45 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:30.289878 17477 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 44 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 44 because replica has last-logged OpId of term: 42 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 42 index: 21.
I20250111 09:01:30.291509 17614 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 45.28, Last appended by leader: 28, Current term: 45, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.291620 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 45 FOLLOWER]: Advancing to term 46
I20250111 09:01:30.291666 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 44 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:30.291847 17760 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 44 FOLLOWER]: Leader election lost for term 44. Reason: could not achieve majority
I20250111 09:01:30.292340 17614 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 46 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 46 because replica has last-logged OpId of term: 45 index: 28, which is greater than that of the candidate, which has last-logged OpId of term: 44 index: 26.
I20250111 09:01:30.292910 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 44 to 45, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 45 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:30.293850 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 46 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:30.293998 17760 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 46 FOLLOWER]: Leader election lost for term 46. Reason: could not achieve majority
W20250111 09:01:30.296314 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.299321 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.299580 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.300386 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.303395 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.304819 17761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.307022 17761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.308260 17761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.310510 17761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.311810 17761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.313776 17761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.317368 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.317386 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.321074 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.321827 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.323957 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.326714 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:30.327188 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:30.327284 17477 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 44 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.327343 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 44 FOLLOWER]: Advancing to term 45
W20250111 09:01:30.327633 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.328107 17477 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 45 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.328418 17477 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 45 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.328666 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 45 candidate_status { last_received { term: 42 index: 23 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.328668 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 45 candidate_status { last_received { term: 42 index: 23 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.328760 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 44 FOLLOWER]: Advancing to term 45
I20250111 09:01:30.328759 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 44 FOLLOWER]: Advancing to term 45
I20250111 09:01:30.328963 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:30.329063 17477 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 46 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.329103 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 46 FOLLOWER]: Advancing to term 47
I20250111 09:01:30.329550 17339 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 45 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 45.
I20250111 09:01:30.329628 17614 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 45 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 45 because replica has last-logged OpId of term: 43 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 42 index: 23.
I20250111 09:01:30.329784 17477 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 47 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.329845 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 45 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:30.330029 17742 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 45 FOLLOWER]: Leader election won for term 45
I20250111 09:01:30.330106 17742 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 45 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.330219 17742 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 42.23, Last appended by leader: 23, Current term: 45, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.330271 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 47 candidate_status { last_received { term: 44 index: 29 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.330345 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 46 FOLLOWER]: Advancing to term 47
I20250111 09:01:30.330330 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 47 candidate_status { last_received { term: 44 index: 29 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.330422 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 46 FOLLOWER]: Advancing to term 47
I20250111 09:01:30.331055 17339 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 47 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 47.
I20250111 09:01:30.331110 17614 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 47 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 47 because replica has last-logged OpId of term: 45 index: 28, which is greater than that of the candidate, which has last-logged OpId of term: 44 index: 29.
I20250111 09:01:30.331234 17477 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 47 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.331338 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 47 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters: b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:30.331481 17742 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 47 FOLLOWER]: Leader election won for term 47
I20250111 09:01:30.331548 17742 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 47 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.331672 17742 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 44.29, Last appended by leader: 29, Current term: 47, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.331728 17180 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 42 to 45. New cstate: current_term: 45 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:30.332974 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.334506 17180 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 45 to 47, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 47 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:30.334890 17761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:30.335000 17614 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 47 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 45 index: 28. Preceding OpId from leader: term: 47 index: 31. (index mismatch)
I20250111 09:01:30.335270 17742 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 27, Time since last communication: 0.000s
I20250111 09:01:30.335368 17742 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 45.28 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 27
I20250111 09:01:30.335748 17339 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 47 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 44 index: 26. Preceding OpId from leader: term: 47 index: 31. (index mismatch)
I20250111 09:01:30.335966 17614 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 27
I20250111 09:01:30.336012 17614 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted NO_OP operation due to leader change: 45.28
I20250111 09:01:30.336043 17614 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 47 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.336163 17742 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 26, Time since last communication: 0.000s
W20250111 09:01:30.342151 17761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.343565 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.343708 17763 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.344597 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.345515 17763 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.345592 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.347175 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.347718 17299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.349400 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.360246 17338 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 45 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 42 index: 21. Preceding OpId from leader: term: 45 index: 25. (index mismatch)
I20250111 09:01:30.360603 17613 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 45 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 43 index: 22. Preceding OpId from leader: term: 45 index: 25. (index mismatch)
I20250111 09:01:30.361058 17744 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 21, Time since last communication: 0.000s
I20250111 09:01:30.361151 17744 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 43.22 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 21
I20250111 09:01:30.362839 17691 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 21, Time since last communication: 0.000s
I20250111 09:01:30.364372 17614 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 21
I20250111 09:01:30.364413 17614 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted NO_OP operation due to leader change: 43.22
I20250111 09:01:30.364440 17614 raft_consensus.cc:2883] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 45 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:30.369823 17756 mvcc.cc:204] Tried to move back new op lower bound from 7113056626117074944 to 7113056625993908224. Current Snapshot: MvccSnapshot[applied={T|T < 7113056626117074944}]
I20250111 09:01:30.410858 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:30.415864 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:30.423658 17613 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 45 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.423746 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 45 FOLLOWER]: Advancing to term 46
I20250111 09:01:30.424839 17613 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 46 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.425379 17613 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 46 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:30.425484 17611 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 46 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 45. Current term is 46. Ops: [45.39-45.40]
I20250111 09:01:30.425959 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:30.425976 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 46 candidate_status { last_received { term: 45 index: 38 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.426062 17611 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 47 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.426088 17477 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 45 LEADER]: Stepping down as leader of term 45
I20250111 09:01:30.426112 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 47 FOLLOWER]: Advancing to term 48
I20250111 09:01:30.426136 17477 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 45 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.426098 17743 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: INVALID_TERM, Last received: 45.38, Next index: 39, Last known committed idx: 35, Time since last communication: 0.001s
I20250111 09:01:30.427000 17611 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 48 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.427069 17477 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 45.40, Last appended by leader: 40, Current term: 45, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.427175 17614 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 48 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 47. Current term is 48. Ops: [47.53-47.53]
I20250111 09:01:30.427201 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 45 FOLLOWER]: Advancing to term 46
I20250111 09:01:30.427295 17611 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 48 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:30.427964 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 46 candidate_status { last_received { term: 45 index: 38 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.428058 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 45 FOLLOWER]: Advancing to term 46
I20250111 09:01:30.428125 17477 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 46 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 46 because replica has last-logged OpId of term: 45 index: 40, which is greater than that of the candidate, which has last-logged OpId of term: 45 index: 38.
I20250111 09:01:30.428696 17742 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: INVALID_TERM, Last received: 47.52, Next index: 53, Last known committed idx: 50, Time since last communication: 0.000s
I20250111 09:01:30.428831 17742 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 47 LEADER]: Stepping down as leader of term 47
I20250111 09:01:30.428869 17742 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 47 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.428926 17338 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 46 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 46 because replica has last-logged OpId of term: 45 index: 39, which is greater than that of the candidate, which has last-logged OpId of term: 45 index: 38.
I20250111 09:01:30.428961 17742 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 50, Committed index: 52, Last appended: 47.53, Last appended by leader: 53, Current term: 47, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.429388 17742 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 47 FOLLOWER]: Advancing to term 48
I20250111 09:01:30.430114 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 48 candidate_status { last_received { term: 47 index: 52 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.430193 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 47 FOLLOWER]: Advancing to term 48
I20250111 09:01:30.430986 17338 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 48 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 48.
I20250111 09:01:30.431910 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 46 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:30.432096 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 48 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:30.432299 17338 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 48 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 47. Current term is 48. Ops: [47.51-47.53]
W20250111 09:01:30.432451 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.432830 17769 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 46 FOLLOWER]: Leader election lost for term 46. Reason: could not achieve majority
I20250111 09:01:30.432916 17769 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 48 FOLLOWER]: Leader election won for term 48
I20250111 09:01:30.432981 17769 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 48 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:30.433121 17769 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 50, Committed index: 50, Last appended: 47.52, Last appended by leader: 52, Current term: 48, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:30.434247 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.434782 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.435528 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.436357 17743 raft_consensus.cc:931] Couldn't advance consensus term.: Illegal state: Can't advance term to: 46 current term: 46 is higher.
I20250111 09:01:30.436435 17180 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 47 to 48, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 48 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:30.437690 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 48 candidate_status { last_received { term: 47 index: 52 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.437852 17477 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 48 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 48 because replica has last-logged OpId of term: 47 index: 53, which is greater than that of the candidate, which has last-logged OpId of term: 47 index: 52.
I20250111 09:01:30.439669 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:30.440563 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:30.440655 17338 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 46 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.440707 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 46 FOLLOWER]: Advancing to term 47
I20250111 09:01:30.441444 17338 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 47 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:30.442468 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.442914 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 47 candidate_status { last_received { term: 45 index: 39 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.443015 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 46 FOLLOWER]: Advancing to term 47
W20250111 09:01:30.443621 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.443884 17476 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 47 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 47 because replica has last-logged OpId of term: 45 index: 40, which is greater than that of the candidate, which has last-logged OpId of term: 45 index: 39.
W20250111 09:01:30.444665 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.444669 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.445248 17338 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 47 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
W20250111 09:01:30.445447 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.445952 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:30.446036 17338 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 48 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.446089 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 48 FOLLOWER]: Advancing to term 49
I20250111 09:01:30.446760 17338 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 49 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:30.447196 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.447098 17338 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 49 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.447371 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 49 candidate_status { last_received { term: 47 index: 50 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.447583 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 48 FOLLOWER]: Advancing to term 49
I20250111 09:01:30.448707 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 49 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 49 because replica has last-logged OpId of term: 47 index: 53, which is greater than that of the candidate, which has last-logged OpId of term: 47 index: 50.
I20250111 09:01:30.449371 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 47 candidate_status { last_received { term: 45 index: 39 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.449470 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 46 FOLLOWER]: Advancing to term 47
I20250111 09:01:30.450259 17613 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 47 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 47.
I20250111 09:01:30.450584 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 49 candidate_status { last_received { term: 47 index: 50 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.450670 17611 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 48 LEADER]: Stepping down as leader of term 48
I20250111 09:01:30.450706 17611 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 48 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:30.450830 17611 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 50, Committed index: 50, Last appended: 48.53, Last appended by leader: 53, Current term: 48, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.450946 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 48 FOLLOWER]: Advancing to term 49
I20250111 09:01:30.451772 17611 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 49 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 49 because replica has last-logged OpId of term: 48 index: 53, which is greater than that of the candidate, which has last-logged OpId of term: 47 index: 50.
W20250111 09:01:30.452629 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.453243 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.453737 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.453737 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.454490 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 47 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:01:30.454722 17760 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 47 FOLLOWER]: Leader election won for term 47
I20250111 09:01:30.454814 17760 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 47 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:30.454931 17760 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 45.39, Last appended by leader: 39, Current term: 47, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.456110 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 49 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:30.456307 17759 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 49 FOLLOWER]: Leader election lost for term 49. Reason: could not achieve majority
I20250111 09:01:30.456213 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 45 to 47, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 47 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:30.456858 17763 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.457674 17764 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.457973 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.458784 17761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.458832 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.458863 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.460772 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.461774 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.462190 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.462918 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.464784 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.465122 17611 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 47 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 45 index: 38. Preceding OpId from leader: term: 47 index: 41. (index mismatch)
I20250111 09:01:30.465142 17476 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 47 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 45 index: 40. Preceding OpId from leader: term: 47 index: 41. (index mismatch)
I20250111 09:01:30.465885 17760 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 39, Time since last communication: 0.000s
I20250111 09:01:30.465963 17760 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer 262623ac6df745c785ce267ea39bd265 log is divergent from this leader: its last log entry 45.40 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 39
I20250111 09:01:30.466444 17476 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 39
I20250111 09:01:30.466491 17476 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 45.40
W20250111 09:01:30.466575 17476 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
I20250111 09:01:30.466684 17760 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 35, Time since last communication: 0.000s
W20250111 09:01:30.468204 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.468472 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.473238 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.473863 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.474431 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.477478 17761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.477679 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.478057 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.484809 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.485631 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.486243 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.486780 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.489913 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.491909 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.492584 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
W20250111 09:01:30.493112 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.493670 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:30.493777 17477 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 47 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.493834 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 47 FOLLOWER]: Advancing to term 48
I20250111 09:01:30.495558 17477 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 48 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.496014 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 48 candidate_status { last_received { term: 47 index: 48 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.496107 17339 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 47 LEADER]: Stepping down as leader of term 47
I20250111 09:01:30.496136 17339 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 47 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:30.496229 17339 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 48, Committed index: 48, Last appended: 47.48, Last appended by leader: 48, Current term: 47, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.496325 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 47 FOLLOWER]: Advancing to term 48
I20250111 09:01:30.497018 17339 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 48 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 48.
I20250111 09:01:30.497267 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 48 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:30.497421 17743 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 48 FOLLOWER]: Leader election won for term 48
I20250111 09:01:30.497486 17743 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 48 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.497591 17743 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 48, Committed index: 48, Last appended: 47.48, Last appended by leader: 48, Current term: 48, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:30.498034 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.498040 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.498790 17761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:30.499528 17180 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 47 to 48, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 48 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:30.500108 17477 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 48 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.500784 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 48 candidate_status { last_received { term: 47 index: 48 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.500883 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 47 FOLLOWER]: Advancing to term 48
W20250111 09:01:30.501446 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.501842 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:30.501992 17611 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 48 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 48.
W20250111 09:01:30.502898 17781 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.503093 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.503829 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:30.503914 17477 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 49 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.503958 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 49 FOLLOWER]: Advancing to term 50
I20250111 09:01:30.504767 17477 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 50 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.505327 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 50 candidate_status { last_received { term: 47 index: 53 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.505424 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 49 FOLLOWER]: Advancing to term 50
I20250111 09:01:30.506276 17339 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 50 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 50.
I20250111 09:01:30.506526 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 50 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:30.506656 17742 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 50 FOLLOWER]: Leader election won for term 50
I20250111 09:01:30.506728 17742 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 50 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.506856 17742 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 52, Committed index: 52, Last appended: 47.53, Last appended by leader: 53, Current term: 50, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:30.507233 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.507524 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.508596 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 48 to 50, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 50 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:30.509073 17477 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 50 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.509093 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 50 candidate_status { last_received { term: 47 index: 53 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.509181 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 49 FOLLOWER]: Advancing to term 50
W20250111 09:01:30.509366 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.509894 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.510370 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:30.509917 17611 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 50 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 50 because replica has last-logged OpId of term: 48 index: 53, which is greater than that of the candidate, which has last-logged OpId of term: 47 index: 53.
W20250111 09:01:30.511083 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.512979 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.513319 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.514871 17339 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 48 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 47 index: 48. Preceding OpId from leader: term: 48 index: 50. (index mismatch)
I20250111 09:01:30.515203 17611 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 48 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 47 index: 48. Preceding OpId from leader: term: 48 index: 50. (index mismatch)
I20250111 09:01:30.515964 17691 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 49, Last known committed idx: 48, Time since last communication: 0.000s
I20250111 09:01:30.516228 17743 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 49, Last known committed idx: 48, Time since last communication: 0.000s
I20250111 09:01:30.516698 17339 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 50 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 47 index: 50. Preceding OpId from leader: term: 50 index: 56. (index mismatch)
I20250111 09:01:30.516827 17611 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 50 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 48 index: 53. Preceding OpId from leader: term: 50 index: 56. (index mismatch)
I20250111 09:01:30.517148 17691 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 54, Last known committed idx: 48, Time since last communication: 0.000s
I20250111 09:01:30.517400 17691 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 54, Last known committed idx: 50, Time since last communication: 0.000s
I20250111 09:01:30.517469 17691 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 48.53 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 50
I20250111 09:01:30.520118 17613 raft_consensus.cc:1212] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 50 FOLLOWER]: Deduplicated request from leader. Original: 47.50->[47.51-50.57] Dedup: 47.52->[47.53-50.57]
I20250111 09:01:30.520193 17613 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 52
I20250111 09:01:30.520219 17613 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted NO_OP operation due to leader change: 48.53
I20250111 09:01:30.520246 17613 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 50 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20250111 09:01:30.521672 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.522086 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.526017 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.542423 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.568380 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:30.568764 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:30.574179 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 48 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.574254 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 48 FOLLOWER]: Advancing to term 49
I20250111 09:01:30.575232 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 49 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.575685 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 49 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:30.576400 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 49 candidate_status { last_received { term: 48 index: 58 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.577924 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:30.578012 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 50 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.578074 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 50 FOLLOWER]: Advancing to term 51
I20250111 09:01:30.578724 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 51 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.578967 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 51 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:30.580088 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 51 candidate_status { last_received { term: 50 index: 79 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.580183 17476 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 50 LEADER]: Stepping down as leader of term 50
I20250111 09:01:30.580219 17476 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 50 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.580332 17476 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 81, Committed index: 81, Last appended: 50.83, Last appended by leader: 83, Current term: 50, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.580504 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:30.580546 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 50 FOLLOWER]: Advancing to term 51
I20250111 09:01:30.580862 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 49 candidate_status { last_received { term: 48 index: 58 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.580952 17477 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 48 LEADER]: Stepping down as leader of term 48
I20250111 09:01:30.580937 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:30.580988 17477 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 48 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.581046 17340 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 48 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.581089 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 48 FOLLOWER]: Advancing to term 49
I20250111 09:01:30.581112 17477 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 56, Committed index: 56, Last appended: 48.64, Last appended by leader: 64, Current term: 48, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.581203 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 48 FOLLOWER]: Advancing to term 49
I20250111 09:01:30.581414 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 51 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 51 because replica has last-logged OpId of term: 50 index: 83, which is greater than that of the candidate, which has last-logged OpId of term: 50 index: 79.
W20250111 09:01:30.581831 17756 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: Replica 262623ac6df745c785ce267ea39bd265 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 51 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
I20250111 09:01:30.581887 17340 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 49 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.582278 17477 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 49 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 49 because replica has last-logged OpId of term: 48 index: 64, which is greater than that of the candidate, which has last-logged OpId of term: 48 index: 58.
I20250111 09:01:30.582453 17340 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 49 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.582515 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 49 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:30.582962 17769 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 49 FOLLOWER]: Leader election lost for term 49. Reason: could not achieve majority
I20250111 09:01:30.583086 17339 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:30.583169 17339 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 50 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.583216 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 50 FOLLOWER]: Advancing to term 51
W20250111 09:01:30.583218 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.583599 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 49 candidate_status { last_received { term: 48 index: 58 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.583721 17477 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 49 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 49 because replica has last-logged OpId of term: 48 index: 64, which is greater than that of the candidate, which has last-logged OpId of term: 48 index: 58.
I20250111 09:01:30.584079 17339 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 51 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.584110 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 51 candidate_status { last_received { term: 50 index: 79 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.584329 17340 raft_consensus.cc:2388] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 51 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c in current term 51: Already voted for candidate fb270e42d7c9400f8c05bb9565db1972 in this term.
W20250111 09:01:30.584939 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.584398 17339 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 51 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.585659 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 51 candidate_status { last_received { term: 50 index: 83 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.586380 17477 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 51 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 51.
I20250111 09:01:30.586653 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 51 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:30.586815 17758 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 51 FOLLOWER]: Leader election won for term 51
I20250111 09:01:30.586831 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 51 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:30.586898 17758 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 51 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:30.586922 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 51 candidate_status { last_received { term: 50 index: 83 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.587033 17614 raft_consensus.cc:2388] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 51 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 in current term 51: Already voted for candidate b5fb421fa3354606aecbeafe55a0565c in this term.
I20250111 09:01:30.587029 17758 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 81, Committed index: 81, Last appended: 50.83, Last appended by leader: 83, Current term: 51, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.587180 17769 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 51 FOLLOWER]: Leader election lost for term 51. Reason: could not achieve majority
I20250111 09:01:30.588404 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 49 candidate_status { last_received { term: 48 index: 58 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.588510 17611 raft_consensus.cc:2388] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 49 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 in current term 49: Already voted for candidate b5fb421fa3354606aecbeafe55a0565c in this term.
I20250111 09:01:30.588933 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 50 to 51, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 51 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:30.589605 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 49 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
W20250111 09:01:30.589895 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:30.590016 17699 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 49 FOLLOWER]: Leader election lost for term 49. Reason: could not achieve majority
W20250111 09:01:30.592224 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.592965 17611 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 51 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 50 index: 79. Preceding OpId from leader: term: 51 index: 85. (index mismatch)
I20250111 09:01:30.592989 17477 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 51 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 50 index: 83. Preceding OpId from leader: term: 51 index: 85. (index mismatch)
I20250111 09:01:30.593398 17699 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 84, Last known committed idx: 78, Time since last communication: 0.000s
I20250111 09:01:30.593719 17699 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 84, Last known committed idx: 81, Time since last communication: 0.000s
W20250111 09:01:30.598340 17781 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.603497 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.607249 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.611696 17761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.612334 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.615217 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.616106 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.616756 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.617864 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.620664 17781 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.620896 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.625037 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.625188 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.625576 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.627030 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.631795 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.633234 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.634323 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.635800 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.641386 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.642539 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.644546 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.646662 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.649201 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.651249 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.656908 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.657505 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.661128 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.663632 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
W20250111 09:01:30.663617 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.664101 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:30.664217 17477 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 49 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.664268 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 49 FOLLOWER]: Advancing to term 50
I20250111 09:01:30.665016 17477 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 50 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.665324 17477 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 50 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.665630 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 50 candidate_status { last_received { term: 48 index: 64 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.665738 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 49 FOLLOWER]: Advancing to term 50
I20250111 09:01:30.665732 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 50 candidate_status { last_received { term: 48 index: 64 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.665890 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 49 FOLLOWER]: Advancing to term 50
I20250111 09:01:30.666549 17339 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 50 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 50.
I20250111 09:01:30.666725 17611 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 50 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 50.
I20250111 09:01:30.666777 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 50 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:30.666914 17743 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 50 FOLLOWER]: Leader election won for term 50
I20250111 09:01:30.666998 17743 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 50 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.667119 17743 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 56, Committed index: 56, Last appended: 48.64, Last appended by leader: 64, Current term: 50, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:30.667512 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.668195 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:30.668277 17477 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 51 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.668330 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 51 FOLLOWER]: Advancing to term 52
W20250111 09:01:30.668654 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:30.669145 17477 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 52 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.669412 17477 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 52 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.669363 17178 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 48 to 50. New cstate: current_term: 50 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:30.669699 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 52 candidate_status { last_received { term: 51 index: 90 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.669698 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 52 candidate_status { last_received { term: 51 index: 90 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.669783 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 51 FOLLOWER]: Advancing to term 52
I20250111 09:01:30.669783 17339 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 51 LEADER]: Stepping down as leader of term 51
I20250111 09:01:30.669829 17339 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 51 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:30.669950 17339 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 90, Committed index: 90, Last appended: 51.90, Last appended by leader: 90, Current term: 51, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.670075 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 51 FOLLOWER]: Advancing to term 52
I20250111 09:01:30.670500 17611 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 52 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 52.
I20250111 09:01:30.670713 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 52 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:30.670855 17695 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 52 FOLLOWER]: Leader election won for term 52
I20250111 09:01:30.670943 17695 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 52 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.671041 17695 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 90, Committed index: 90, Last appended: 51.90, Last appended by leader: 90, Current term: 52, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.671540 17339 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 52 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 52.
W20250111 09:01:30.672610 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:30.672904 17178 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 51 to 52, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 52 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:30.674160 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.680411 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:30.682269 17339 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 50 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 48 index: 58. Preceding OpId from leader: term: 50 index: 66. (index mismatch)
I20250111 09:01:30.682269 17611 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 50 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 48 index: 58. Preceding OpId from leader: term: 50 index: 66. (index mismatch)
I20250111 09:01:30.682610 17743 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 65, Last known committed idx: 56, Time since last communication: 0.000s
I20250111 09:01:30.682727 17695 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 65, Last known committed idx: 56, Time since last communication: 0.000s
I20250111 09:01:30.690011 17730 mvcc.cc:204] Tried to move back new op lower bound from 7113056627431575552 to 7113056627373682688. Current Snapshot: MvccSnapshot[applied={T|T < 7113056627007602688}]
W20250111 09:01:30.695376 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.696415 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.697724 17340 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 52 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 51 index: 90. Preceding OpId from leader: term: 52 index: 92. (index mismatch)
I20250111 09:01:30.697885 17614 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 52 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 51 index: 90. Preceding OpId from leader: term: 52 index: 92. (index mismatch)
I20250111 09:01:30.698828 17743 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 91, Last known committed idx: 90, Time since last communication: 0.000s
I20250111 09:01:30.699872 17743 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 91, Last known committed idx: 90, Time since last communication: 0.000s
W20250111 09:01:30.701805 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.735857 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:30.736647 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:30.736760 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 50 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.736812 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 50 FOLLOWER]: Advancing to term 51
I20250111 09:01:30.737756 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 51 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.738090 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 51 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:30.738500 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 51 candidate_status { last_received { term: 50 index: 78 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.738600 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 50 FOLLOWER]: Advancing to term 51
I20250111 09:01:30.739310 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:30.739410 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 52 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.739463 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 52 FOLLOWER]: Advancing to term 53
I20250111 09:01:30.739468 17339 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 51 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 51 because replica has last-logged OpId of term: 50 index: 81, which is greater than that of the candidate, which has last-logged OpId of term: 50 index: 78.
I20250111 09:01:30.740401 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 53 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.740697 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 53 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:30.741300 17339 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 51 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 50. Current term is 51. Ops: [50.82-50.82]
I20250111 09:01:30.741519 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 51 candidate_status { last_received { term: 50 index: 78 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.741607 17614 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 53 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 52. Current term is 53. Ops: [52.99-52.101]
I20250111 09:01:30.741704 17477 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 50 LEADER]: Stepping down as leader of term 50
I20250111 09:01:30.741783 17477 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 50 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.741943 17743 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: INVALID_TERM, Last received: 52.98, Next index: 99, Last known committed idx: 97, Time since last communication: 0.000s
I20250111 09:01:30.741986 17611 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 51 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 50. Current term is 51. Ops: [50.79-50.80]
I20250111 09:01:30.741966 17477 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 81, Committed index: 81, Last appended: 50.82, Last appended by leader: 82, Current term: 50, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.742101 17744 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 52 LEADER]: Stepping down as leader of term 52
I20250111 09:01:30.742278 17744 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 52 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.742440 17744 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 98, Committed index: 98, Last appended: 52.102, Last appended by leader: 102, Current term: 52, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.742548 17744 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 52 FOLLOWER]: Advancing to term 53
W20250111 09:01:30.743093 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.743968 17743 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: INVALID_TERM, Last received: 50.81, Next index: 82, Last known committed idx: 81, Time since last communication: 0.000s
I20250111 09:01:30.744092 17695 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: INVALID_TERM, Last received: 50.78, Next index: 79, Last known committed idx: 77, Time since last communication: 0.000s
I20250111 09:01:30.744284 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 50 FOLLOWER]: Advancing to term 51
I20250111 09:01:30.744446 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 53 candidate_status { last_received { term: 52 index: 98 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.744558 17467 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 53 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 53 because replica has last-logged OpId of term: 52 index: 102, which is greater than that of the candidate, which has last-logged OpId of term: 52 index: 98.
I20250111 09:01:30.745461 17477 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 51 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 51 because replica has last-logged OpId of term: 50 index: 82, which is greater than that of the candidate, which has last-logged OpId of term: 50 index: 78.
I20250111 09:01:30.745505 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 53 candidate_status { last_received { term: 52 index: 98 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.745615 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 52 FOLLOWER]: Advancing to term 53
I20250111 09:01:30.746500 17338 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 53 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 53 because replica has last-logged OpId of term: 52 index: 101, which is greater than that of the candidate, which has last-logged OpId of term: 52 index: 98.
W20250111 09:01:30.747699 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.748104 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 51 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:30.748252 17723 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 51 FOLLOWER]: Leader election lost for term 51. Reason: could not achieve majority
W20250111 09:01:30.748294 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.748893 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 53 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:30.749054 17723 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 53 FOLLOWER]: Leader election lost for term 53. Reason: could not achieve majority
W20250111 09:01:30.752368 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.752403 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.754307 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.754312 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.754977 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.756402 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.759199 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.760376 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.762634 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.763057 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.763855 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.764402 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.765311 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.770315 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.770596 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.771445 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.772100 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.773231 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.778328 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.779023 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.779054 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.781543 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.781749 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.787213 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.788246 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.788494 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.790637 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.793370 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.793402 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.797940 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.799427 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.800635 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.802992 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.805949 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.806993 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.809456 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.812453 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.812633 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.817153 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.819231 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.821964 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.823601 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.823633 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.831419 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.832892 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.833036 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:30.833439 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:30.833562 17338 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 51 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.833654 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 51 FOLLOWER]: Advancing to term 52
I20250111 09:01:30.834473 17338 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 52 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.834817 17338 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 52 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.835178 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 52 candidate_status { last_received { term: 50 index: 81 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.835333 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:30.835428 17338 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 53 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.835492 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 53 FOLLOWER]: Advancing to term 54
I20250111 09:01:30.835481 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 51 FOLLOWER]: Advancing to term 52
I20250111 09:01:30.836311 17338 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 54 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.836395 17477 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 52 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 52 because replica has last-logged OpId of term: 50 index: 82, which is greater than that of the candidate, which has last-logged OpId of term: 50 index: 81.
W20250111 09:01:30.836376 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.836760 17338 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 54 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.837018 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 54 candidate_status { last_received { term: 52 index: 101 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.837109 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 53 FOLLOWER]: Advancing to term 54
W20250111 09:01:30.837544 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.838045 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 54 candidate_status { last_received { term: 52 index: 101 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.838147 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 53 FOLLOWER]: Advancing to term 54
W20250111 09:01:30.838632 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:30.839010 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 54 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 54.
I20250111 09:01:30.839205 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 52 candidate_status { last_received { term: 50 index: 81 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.839253 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 54 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:30.839301 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 51 FOLLOWER]: Advancing to term 52
I20250111 09:01:30.839397 17758 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 54 FOLLOWER]: Leader election won for term 54
I20250111 09:01:30.839474 17758 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 54 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:30.839596 17758 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 98, Committed index: 98, Last appended: 52.101, Last appended by leader: 100, Current term: 54, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.840101 17611 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 52 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 52.
I20250111 09:01:30.840305 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 52 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:01:30.840449 17699 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 52 FOLLOWER]: Leader election won for term 52
I20250111 09:01:30.840517 17699 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 52 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:30.840559 17477 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 54 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 54 because replica has last-logged OpId of term: 52 index: 102, which is greater than that of the candidate, which has last-logged OpId of term: 52 index: 101.
I20250111 09:01:30.840603 17699 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 81, Committed index: 81, Last appended: 50.81, Last appended by leader: 81, Current term: 52, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.840790 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 52 to 54, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 54 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:30.843235 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 50 to 52, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 52 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:30.846338 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.846393 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:30.851434 17611 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 54 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 52 index: 98. Preceding OpId from leader: term: 54 index: 103. (index mismatch)
I20250111 09:01:30.851481 17477 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 54 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 52 index: 102. Preceding OpId from leader: term: 54 index: 103. (index mismatch)
I20250111 09:01:30.851835 17758 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 102, Last known committed idx: 97, Time since last communication: 0.000s
I20250111 09:01:30.851953 17699 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 102, Last known committed idx: 98, Time since last communication: 0.000s
I20250111 09:01:30.852017 17699 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer 262623ac6df745c785ce267ea39bd265 log is divergent from this leader: its last log entry 52.102 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 98
W20250111 09:01:30.852434 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:30.853395 17477 raft_consensus.cc:1212] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 54 FOLLOWER]: Deduplicated request from leader. Original: 52.98->[52.99-54.103] Dedup: 52.101->[54.102-54.103]
I20250111 09:01:30.853502 17477 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 101
I20250111 09:01:30.853556 17477 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 52.102
W20250111 09:01:30.853654 17477 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
W20250111 09:01:30.854153 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.862696 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.867013 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.868577 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.871381 17477 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 52 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 50 index: 82. Preceding OpId from leader: term: 52 index: 83. (index mismatch)
I20250111 09:01:30.872594 17759 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 82, Last known committed idx: 81, Time since last communication: 0.000s
I20250111 09:01:30.872737 17759 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer 262623ac6df745c785ce267ea39bd265 log is divergent from this leader: its last log entry 50.82 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 81
I20250111 09:01:30.874059 17476 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 81
I20250111 09:01:30.874110 17476 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 50.82
W20250111 09:01:30.874195 17476 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
I20250111 09:01:30.876048 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:30.877586 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:30.880435 17611 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 52 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 50 index: 78. Preceding OpId from leader: term: 52 index: 83. (index mismatch)
I20250111 09:01:30.881664 17699 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 82, Last known committed idx: 77, Time since last communication: 0.000s
I20250111 09:01:30.885684 17476 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 52 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.885767 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 52 FOLLOWER]: Advancing to term 53
I20250111 09:01:30.886723 17476 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 53 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.887223 17467 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 53 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 52. Current term is 53. Ops: [52.87-52.89]
I20250111 09:01:30.887483 17476 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 53 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.887564 17699 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: INVALID_TERM, Last received: 52.86, Next index: 87, Last known committed idx: 84, Time since last communication: 0.000s
I20250111 09:01:30.887718 17699 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 52 LEADER]: Stepping down as leader of term 52
I20250111 09:01:30.887766 17699 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 52 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:30.887882 17699 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 86, Committed index: 86, Last appended: 52.89, Last appended by leader: 89, Current term: 52, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.888015 17699 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 52 FOLLOWER]: Advancing to term 53
W20250111 09:01:30.888151 17781 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.888473 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 53 candidate_status { last_received { term: 52 index: 86 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
W20250111 09:01:30.889552 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.889910 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 53 candidate_status { last_received { term: 52 index: 86 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.890040 17338 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 53 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 53 because replica has last-logged OpId of term: 52 index: 89, which is greater than that of the candidate, which has last-logged OpId of term: 52 index: 86.
I20250111 09:01:30.890555 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:30.890652 17477 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 54 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.890704 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 54 FOLLOWER]: Advancing to term 55
I20250111 09:01:30.890260 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 53 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:30.890954 17744 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 53 FOLLOWER]: Leader election lost for term 53. Reason: could not achieve majority
W20250111 09:01:30.891324 17781 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.892565 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.892625 17477 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 55 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.892822 17476 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 55 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 54. Current term is 55. Ops: [54.116-54.116]
I20250111 09:01:30.893666 17759 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: INVALID_TERM, Last received: 54.115, Next index: 116, Last known committed idx: 115, Time since last communication: 0.000s
I20250111 09:01:30.893800 17759 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 54 LEADER]: Stepping down as leader of term 54
I20250111 09:01:30.893848 17759 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 54 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:30.893962 17759 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 115, Committed index: 115, Last appended: 54.116, Last appended by leader: 116, Current term: 54, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.894116 17759 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 54 FOLLOWER]: Advancing to term 55
W20250111 09:01:30.895087 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.897248 17477 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 55 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:30.897642 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 55 candidate_status { last_received { term: 54 index: 115 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:30.897758 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 54 FOLLOWER]: Advancing to term 55
W20250111 09:01:30.898312 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.898474 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 55 candidate_status { last_received { term: 54 index: 115 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.898602 17338 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 55 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 55 because replica has last-logged OpId of term: 54 index: 116, which is greater than that of the candidate, which has last-logged OpId of term: 54 index: 115.
I20250111 09:01:30.898660 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 55 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 55.
W20250111 09:01:30.898711 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.899047 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 55 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:30.899196 17744 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 55 FOLLOWER]: Leader election won for term 55
I20250111 09:01:30.899267 17744 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 55 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.899379 17744 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 115, Committed index: 115, Last appended: 54.115, Last appended by leader: 115, Current term: 55, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:30.899549 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.900853 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 54 to 55, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 55 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:30.901582 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.903245 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.903878 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:30.904932 17339 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 55 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 54 index: 116. Preceding OpId from leader: term: 55 index: 117. (index mismatch)
I20250111 09:01:30.905318 17614 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 55 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 54 index: 115. Preceding OpId from leader: term: 55 index: 117. (index mismatch)
I20250111 09:01:30.905356 17744 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 116, Last known committed idx: 115, Time since last communication: 0.000s
I20250111 09:01:30.905437 17744 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer fb270e42d7c9400f8c05bb9565db1972 log is divergent from this leader: its last log entry 54.116 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 115
I20250111 09:01:30.905644 17743 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 116, Last known committed idx: 115, Time since last communication: 0.000s
I20250111 09:01:30.906075 17339 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 115
I20250111 09:01:30.906117 17339 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted WRITE_OP operation due to leader change: 54.116
W20250111 09:01:30.906206 17339 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Aborted: Op aborted by new leader
W20250111 09:01:30.907157 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.907920 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.910812 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.913615 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.914315 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.914397 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.915323 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.919783 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.922212 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.922212 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.923185 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.923478 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.924489 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.927546 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.928470 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.930387 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.930783 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.931586 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.932103 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.933378 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.935177 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.936391 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.937495 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.941769 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.941900 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.942833 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.943140 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.945163 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.946058 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.947963 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.949416 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.950523 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.952342 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.953783 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.955331 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.956645 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.957547 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.960822 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.963557 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.964480 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.964915 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.966117 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.968631 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.970335 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.975320 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.976038 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.976593 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.978648 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:30.979105 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:30.979228 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 52 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.979295 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 52 FOLLOWER]: Advancing to term 53
I20250111 09:01:30.980341 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 53 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:30.980928 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.980940 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 53 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:30.980968 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 53 candidate_status { last_received { term: 52 index: 88 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.981138 17467 raft_consensus.cc:2388] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 53 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c in current term 53: Already voted for candidate 262623ac6df745c785ce267ea39bd265 in this term.
I20250111 09:01:30.981488 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:30.981597 17611 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 55 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:30.981665 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 55 FOLLOWER]: Advancing to term 56
W20250111 09:01:30.982524 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.982563 17611 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 56 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.982897 17611 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 56 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:30.983311 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 56 candidate_status { last_received { term: 55 index: 121 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:30.983322 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 53 candidate_status { last_received { term: 52 index: 88 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.983422 17467 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 55 LEADER]: Stepping down as leader of term 55
I20250111 09:01:30.983459 17339 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 53 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 53 because replica has last-logged OpId of term: 52 index: 89, which is greater than that of the candidate, which has last-logged OpId of term: 52 index: 88.
I20250111 09:01:30.983469 17467 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 55 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:30.983528 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 56 candidate_status { last_received { term: 55 index: 121 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:30.983619 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 55 FOLLOWER]: Advancing to term 56
I20250111 09:01:30.983603 17467 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 121, Committed index: 121, Last appended: 55.121, Last appended by leader: 121, Current term: 55, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:30.983789 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 55 FOLLOWER]: Advancing to term 56
I20250111 09:01:30.984534 17338 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 56 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 56.
I20250111 09:01:30.984745 17467 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 56 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 56.
I20250111 09:01:30.984892 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 53 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
W20250111 09:01:30.984966 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.985041 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 56 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:30.985142 17768 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 53 FOLLOWER]: Leader election lost for term 53. Reason: could not achieve majority
I20250111 09:01:30.985415 17768 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 56 FOLLOWER]: Leader election won for term 56
I20250111 09:01:30.985513 17768 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 56 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:30.985648 17768 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 121, Committed index: 121, Last appended: 55.121, Last appended by leader: 121, Current term: 56, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:30.985877 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:30.986977 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 55 to 56, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 56 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
W20250111 09:01:30.987821 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.989475 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:30.990520 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.991921 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.993322 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:30.997826 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.000890 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.002147 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.003510 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.003613 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.005913 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.007086 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.013511 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.014303 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:31.015476 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
W20250111 09:01:31.015755 17738 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:31.015920 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:31.016032 17338 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 53 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:31.016093 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 53 FOLLOWER]: Advancing to term 54
I20250111 09:01:31.016904 17338 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 54 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.017204 17338 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 54 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:31.017514 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 54 candidate_status { last_received { term: 52 index: 89 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
W20250111 09:01:31.017560 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:31.017607 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 53 FOLLOWER]: Advancing to term 54
I20250111 09:01:31.017597 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 54 candidate_status { last_received { term: 52 index: 89 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:31.017684 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 53 FOLLOWER]: Advancing to term 54
I20250111 09:01:31.018039 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:31.018128 17338 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 56 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:31.018182 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 56 FOLLOWER]: Advancing to term 57
I20250111 09:01:31.018450 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 54 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 54.
I20250111 09:01:31.018450 17467 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 54 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 54.
I20250111 09:01:31.018741 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 54 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:31.018913 17699 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 54 FOLLOWER]: Leader election won for term 54
I20250111 09:01:31.018918 17338 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 57 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.019012 17699 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 54 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:31.019153 17699 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 86, Committed index: 86, Last appended: 52.89, Last appended by leader: 89, Current term: 54, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.019251 17338 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 57 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:31.019603 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 57 candidate_status { last_received { term: 55 index: 121 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:31.019716 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 56 FOLLOWER]: Advancing to term 57
I20250111 09:01:31.019795 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 57 candidate_status { last_received { term: 55 index: 121 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:31.019891 17614 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 56 LEADER]: Stepping down as leader of term 56
I20250111 09:01:31.019938 17614 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 56 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:31.020073 17614 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 121, Committed index: 121, Last appended: 56.122, Last appended by leader: 122, Current term: 56, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.020205 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 56 FOLLOWER]: Advancing to term 57
I20250111 09:01:31.020459 17467 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 57 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 57.
I20250111 09:01:31.020524 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 52 to 54. New cstate: current_term: 54 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:31.020704 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 57 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:31.020824 17699 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 57 FOLLOWER]: Leader election won for term 57
I20250111 09:01:31.020907 17699 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 57 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:31.021023 17699 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 121, Committed index: 121, Last appended: 55.121, Last appended by leader: 121, Current term: 57, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:31.021313 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:31.021595 17614 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 57 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 57 because replica has last-logged OpId of term: 56 index: 122, which is greater than that of the candidate, which has last-logged OpId of term: 55 index: 121.
I20250111 09:01:31.023937 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 56 to 57, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 57 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:31.025262 17467 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 54 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 52 index: 86. Preceding OpId from leader: term: 54 index: 91. (index mismatch)
I20250111 09:01:31.025601 17759 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 90, Last known committed idx: 84, Time since last communication: 0.000s
I20250111 09:01:31.025707 17614 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 54 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 52 index: 88. Preceding OpId from leader: term: 54 index: 91. (index mismatch)
I20250111 09:01:31.026049 17759 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 90, Last known committed idx: 84, Time since last communication: 0.000s
W20250111 09:01:31.028128 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.033533 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.036703 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.040174 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.041823 17767 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.043905 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.043965 17761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:31.044260 17476 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 57 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 55 index: 121. Preceding OpId from leader: term: 57 index: 123. (index mismatch)
I20250111 09:01:31.044665 17699 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 122, Last known committed idx: 121, Time since last communication: 0.000s
I20250111 09:01:31.048238 17611 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 57 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 56 index: 122. Preceding OpId from leader: term: 57 index: 123. (index mismatch)
I20250111 09:01:31.049081 17760 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 122, Last known committed idx: 121, Time since last communication: 0.000s
I20250111 09:01:31.049162 17760 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 56.122 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 121
I20250111 09:01:31.050025 17611 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 121
I20250111 09:01:31.050073 17611 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted NO_OP operation due to leader change: 56.122
I20250111 09:01:31.050100 17611 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 57 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:31.050474 17728 mvcc.cc:204] Tried to move back new op lower bound from 7113056628913442816 to 7113056628823158784. Current Snapshot: MvccSnapshot[applied={T|T < 7113056628913442816}]
W20250111 09:01:31.061666 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.062346 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:31.122591 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:31.124229 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:31.124346 17476 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 54 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:31.124416 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 54 FOLLOWER]: Advancing to term 55
I20250111 09:01:31.125358 17476 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 55 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.125674 17476 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 55 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:31.126016 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 55 candidate_status { last_received { term: 54 index: 116 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:31.126266 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 55 candidate_status { last_received { term: 54 index: 116 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:31.126366 17338 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 54 LEADER]: Stepping down as leader of term 54
I20250111 09:01:31.126411 17338 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 54 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:31.126549 17338 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 116, Committed index: 116, Last appended: 54.118, Last appended by leader: 118, Current term: 54, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.126644 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 54 FOLLOWER]: Advancing to term 55
I20250111 09:01:31.127511 17338 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 55 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 55 because replica has last-logged OpId of term: 54 index: 118, which is greater than that of the candidate, which has last-logged OpId of term: 54 index: 116.
I20250111 09:01:31.127775 17467 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 55 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 54. Current term is 55. Ops: [54.117-54.118]
I20250111 09:01:31.128062 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 55 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:31.128181 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:31.128253 17467 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 57 FOLLOWER]: Starting forced leader election (received explicit request)
W20250111 09:01:31.128264 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:31.128307 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 57 FOLLOWER]: Advancing to term 58
I20250111 09:01:31.129092 17467 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 58 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.129354 17467 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 58 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:31.129489 17743 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 55 FOLLOWER]: Leader election lost for term 55. Reason: could not achieve majority
I20250111 09:01:31.130816 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 58 candidate_status { last_received { term: 57 index: 142 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:31.131690 17476 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 58 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 57. Current term is 58. Ops: [57.143-57.144]
I20250111 09:01:31.131237 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 58 candidate_status { last_received { term: 57 index: 142 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:31.132172 17338 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 57 LEADER]: Stepping down as leader of term 57
I20250111 09:01:31.132218 17338 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 57 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:31.132335 17338 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 142, Committed index: 142, Last appended: 57.146, Last appended by leader: 146, Current term: 57, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.132426 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 57 FOLLOWER]: Advancing to term 58
W20250111 09:01:31.132720 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:31.134197 17338 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 58 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 58 because replica has last-logged OpId of term: 57 index: 146, which is greater than that of the candidate, which has last-logged OpId of term: 57 index: 142.
I20250111 09:01:31.134445 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 58 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:31.134603 17743 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 58 FOLLOWER]: Leader election lost for term 58. Reason: could not achieve majority
W20250111 09:01:31.136665 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.138360 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.139699 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.140089 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.145776 17761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.146977 17761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.147039 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.153656 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.154757 17748 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 77 similar messages]
W20250111 09:01:31.154857 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.156352 17299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.163034 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.163101 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.165207 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.169945 17761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.169965 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.175715 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.180716 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.181386 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.183632 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.191306 17751 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 49 similar messages]
W20250111 09:01:31.191376 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.192340 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.193778 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.201496 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.202740 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.206377 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.215013 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.215436 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.221378 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.230072 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.230119 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.236290 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.244784 17762 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 86 similar messages]
W20250111 09:01:31.244896 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.247417 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.251916 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.261229 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.263728 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.267907 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.278681 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.280745 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.286962 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.297577 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.301126 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.304675 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.317610 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.320276 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.325994 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:31.337045 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:31.337452 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:31.337548 17611 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 54 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:31.337610 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 54 FOLLOWER]: Advancing to term 55
I20250111 09:01:31.338441 17611 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 55 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.338775 17611 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 55 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:31.339097 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 55 candidate_status { last_received { term: 54 index: 114 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:31.339141 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 55 candidate_status { last_received { term: 54 index: 114 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:31.339236 17467 raft_consensus.cc:2388] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 55 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c in current term 55: Already voted for candidate 262623ac6df745c785ce267ea39bd265 in this term.
I20250111 09:01:31.339290 17338 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 55 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 55 because replica has last-logged OpId of term: 54 index: 118, which is greater than that of the candidate, which has last-logged OpId of term: 54 index: 114.
I20250111 09:01:31.339354 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:31.339440 17611 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 57 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:31.339495 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 57 FOLLOWER]: Advancing to term 58
I20250111 09:01:31.339526 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 55 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:31.339675 17768 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 55 FOLLOWER]: Leader election lost for term 55. Reason: could not achieve majority
I20250111 09:01:31.340130 17611 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 58 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:31.340307 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:31.340421 17611 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 58 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:31.340657 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 58 candidate_status { last_received { term: 57 index: 142 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:31.340698 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 58 candidate_status { last_received { term: 57 index: 142 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:31.340773 17467 raft_consensus.cc:2388] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 58 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c in current term 58: Already voted for candidate 262623ac6df745c785ce267ea39bd265 in this term.
I20250111 09:01:31.340827 17338 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 58 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 58 because replica has last-logged OpId of term: 57 index: 146, which is greater than that of the candidate, which has last-logged OpId of term: 57 index: 142.
I20250111 09:01:31.341068 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 58 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:31.341209 17768 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 58 FOLLOWER]: Leader election lost for term 58. Reason: could not achieve majority
W20250111 09:01:31.342926 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.346922 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.362632 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.363754 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.368356 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.384682 17299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.385717 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.392460 17299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.408417 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.410442 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.413455 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:31.430768 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
W20250111 09:01:31.431246 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:31.431295 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:31.431403 17338 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 55 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:31.431474 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 55 FOLLOWER]: Advancing to term 56
I20250111 09:01:31.432271 17338 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 56 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.432548 17338 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 56 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:31.432832 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 56 candidate_status { last_received { term: 54 index: 118 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:31.432832 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 56 candidate_status { last_received { term: 54 index: 118 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:31.432932 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 55 FOLLOWER]: Advancing to term 56
I20250111 09:01:31.432957 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 55 FOLLOWER]: Advancing to term 56
I20250111 09:01:31.432996 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:31.433104 17338 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 58 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:31.433151 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 58 FOLLOWER]: Advancing to term 59
I20250111 09:01:31.433832 17467 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 56 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 56.
I20250111 09:01:31.433827 17611 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 56 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 56.
I20250111 09:01:31.434054 17338 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 59 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.434108 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 56 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:31.434356 17338 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 59 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:31.434372 17786 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 56 FOLLOWER]: Leader election won for term 56
I20250111 09:01:31.434502 17786 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 56 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
W20250111 09:01:31.434597 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:31.434639 17786 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 116, Committed index: 116, Last appended: 54.118, Last appended by leader: 118, Current term: 56, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.435034 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 59 candidate_status { last_received { term: 57 index: 146 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:31.435137 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 58 FOLLOWER]: Advancing to term 59
I20250111 09:01:31.435678 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 59 candidate_status { last_received { term: 57 index: 146 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:31.435787 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 58 FOLLOWER]: Advancing to term 59
I20250111 09:01:31.435889 17611 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 59 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 59.
I20250111 09:01:31.435999 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 54 to 56. New cstate: current_term: 56 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:31.436111 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 59 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:31.436309 17786 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 59 FOLLOWER]: Leader election won for term 59
I20250111 09:01:31.436393 17786 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 59 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:31.436514 17786 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 142, Committed index: 142, Last appended: 57.146, Last appended by leader: 146, Current term: 59, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.436553 17467 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 59 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 59.
W20250111 09:01:31.439268 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:31.439188 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 57 to 59. New cstate: current_term: 59 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:31.457208 17467 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 59 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 57 index: 142. Preceding OpId from leader: term: 59 index: 148. (index mismatch)
I20250111 09:01:31.457226 17611 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 59 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 57 index: 142. Preceding OpId from leader: term: 59 index: 148. (index mismatch)
I20250111 09:01:31.457515 17786 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 147, Last known committed idx: 140, Time since last communication: 0.000s
I20250111 09:01:31.457613 17759 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 147, Last known committed idx: 140, Time since last communication: 0.000s
I20250111 09:01:31.471431 17467 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 56 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 54 index: 116. Preceding OpId from leader: term: 56 index: 120. (index mismatch)
I20250111 09:01:31.471520 17611 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 56 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 54 index: 114. Preceding OpId from leader: term: 56 index: 120. (index mismatch)
I20250111 09:01:31.471870 17758 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 119, Last known committed idx: 115, Time since last communication: 0.000s
I20250111 09:01:31.471961 17759 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 119, Last known committed idx: 113, Time since last communication: 0.000s
I20250111 09:01:31.649704 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:31.650261 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:31.650388 17476 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 56 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:31.650449 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 56 FOLLOWER]: Advancing to term 57
I20250111 09:01:31.651423 17476 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 57 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.651746 17476 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 57 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:31.652024 17477 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 57 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 56. Current term is 57. Ops: []
I20250111 09:01:31.652448 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 57 candidate_status { last_received { term: 56 index: 174 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:31.652583 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 57 candidate_status { last_received { term: 56 index: 174 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:31.652683 17339 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 56 LEADER]: Stepping down as leader of term 56
I20250111 09:01:31.652717 17339 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 56 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:31.652823 17339 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 174, Committed index: 174, Last appended: 56.174, Last appended by leader: 174, Current term: 56, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.652916 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 56 FOLLOWER]: Advancing to term 57
W20250111 09:01:31.653301 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:31.654297 17339 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 57 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 57.
I20250111 09:01:31.654553 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 57 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters: b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:31.654757 17743 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 57 FOLLOWER]: Leader election won for term 57
I20250111 09:01:31.654914 17743 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 57 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:31.655033 17743 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 173, Committed index: 173, Last appended: 56.174, Last appended by leader: 174, Current term: 57, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.656159 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:31.656358 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 56 to 57, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 57 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:31.657893 17476 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 59 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:31.657960 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 59 FOLLOWER]: Advancing to term 60
I20250111 09:01:31.658686 17476 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 60 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.659138 17476 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 60 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:31.659323 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 60 candidate_status { last_received { term: 59 index: 198 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:31.659958 17339 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 59 LEADER]: Stepping down as leader of term 59
I20250111 09:01:31.660013 17339 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 59 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:31.660125 17339 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 203, Committed index: 203, Last appended: 59.203, Last appended by leader: 203, Current term: 59, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.660751 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 59 FOLLOWER]: Advancing to term 60
I20250111 09:01:31.661549 17339 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 60 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 60 because replica has last-logged OpId of term: 59 index: 203, which is greater than that of the candidate, which has last-logged OpId of term: 59 index: 198.
W20250111 09:01:31.661650 17299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.662631 17299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:31.663057 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 60 candidate_status { last_received { term: 59 index: 198 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:31.663153 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 59 FOLLOWER]: Advancing to term 60
I20250111 09:01:31.664129 17614 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 60 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 60 because replica has last-logged OpId of term: 59 index: 203, which is greater than that of the candidate, which has last-logged OpId of term: 59 index: 198.
W20250111 09:01:31.664709 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:31.664753 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 60 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:31.664907 17743 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 60 FOLLOWER]: Leader election lost for term 60. Reason: could not achieve majority
I20250111 09:01:31.665089 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 56 FOLLOWER]: Advancing to term 57
I20250111 09:01:31.665352 17338 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 57 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 56 index: 174. Preceding OpId from leader: term: 57 index: 177. (index mismatch)
I20250111 09:01:31.665692 17793 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 175, Last known committed idx: 174, Time since last communication: 0.000s
I20250111 09:01:31.665920 17614 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 57 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 56 index: 174. Preceding OpId from leader: term: 57 index: 177. (index mismatch)
I20250111 09:01:31.666181 17793 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 175, Last known committed idx: 173, Time since last communication: 0.000s
W20250111 09:01:31.672585 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.672672 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.673063 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.674607 17794 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.677597 17794 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.678728 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.681746 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.683195 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.686498 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.690439 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.691381 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.692402 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.692581 17299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.693531 17762 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.694180 17299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.695297 17801 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.695528 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.695528 17299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.696655 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.698015 17299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.698500 17781 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.700680 17748 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.701895 17781 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.702073 17794 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.702131 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.702176 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.703476 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.703531 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.705861 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.706882 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.707827 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.708454 17802 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.709797 17299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:31.710156 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:31.710541 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:31.710644 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 57 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:31.710696 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 57 FOLLOWER]: Advancing to term 58
W20250111 09:01:31.711051 17802 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:31.711407 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 58 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.711668 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 58 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:31.711961 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 58 candidate_status { last_received { term: 57 index: 186 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:31.712085 17477 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 57 LEADER]: Stepping down as leader of term 57
I20250111 09:01:31.712150 17477 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 57 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:31.712988 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 58 candidate_status { last_received { term: 57 index: 186 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:31.713152 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 57 FOLLOWER]: Advancing to term 58
I20250111 09:01:31.714016 17338 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 58 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 58.
W20250111 09:01:31.714056 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:31.712296 17477 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 186, Committed index: 186, Last appended: 57.186, Last appended by leader: 186, Current term: 57, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.714516 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 58 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:31.714566 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:31.714666 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 60 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:31.714730 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 60 FOLLOWER]: Advancing to term 61
I20250111 09:01:31.714849 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 57 FOLLOWER]: Advancing to term 58
W20250111 09:01:31.714975 17781 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.715225 17801 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:31.715293 17803 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:31.715240 17789 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 58 FOLLOWER]: Leader election won for term 58
I20250111 09:01:31.715472 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 61 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.715749 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 61 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
W20250111 09:01:31.715659 17802 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:31.715950 17789 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 58 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:31.715952 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 61 candidate_status { last_received { term: 59 index: 203 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:31.716037 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 60 FOLLOWER]: Advancing to term 61
I20250111 09:01:31.716094 17475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 61 candidate_status { last_received { term: 59 index: 203 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:31.716116 17789 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 186, Committed index: 186, Last appended: 57.186, Last appended by leader: 186, Current term: 58, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.716181 17475 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 60 FOLLOWER]: Advancing to term 61
I20250111 09:01:31.716727 17338 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 61 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 61.
I20250111 09:01:31.716980 17475 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 61 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 61.
I20250111 09:01:31.717095 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 61 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:31.717121 17477 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 58 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 58.
I20250111 09:01:31.717365 17789 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 61 FOLLOWER]: Leader election won for term 61
I20250111 09:01:31.717509 17789 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 61 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:31.717469 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 57 to 58, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 58 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
W20250111 09:01:31.718494 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.718565 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:31.717636 17789 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 197, Committed index: 197, Last appended: 59.203, Last appended by leader: 203, Current term: 61, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:31.720778 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 59 to 61, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 61 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
W20250111 09:01:31.721088 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.721434 17751 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.722239 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.722929 17794 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.723663 17781 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.727406 17794 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.728343 17750 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:31.728340 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:32.025055 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:32.025652 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:32.025779 17338 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 58 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:32.025861 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 58 FOLLOWER]: Advancing to term 59
I20250111 09:01:32.026605 17338 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 59 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:32.026935 17338 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 59 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:32.027251 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 59 candidate_status { last_received { term: 57 index: 186 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:32.027369 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 58 FOLLOWER]: Advancing to term 59
I20250111 09:01:32.027338 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 59 candidate_status { last_received { term: 57 index: 186 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:32.027462 17614 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 58 LEADER]: Stepping down as leader of term 58
I20250111 09:01:32.027504 17614 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 58 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:32.027576 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:32.027660 17338 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 61 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:32.027707 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 61 FOLLOWER]: Advancing to term 62
I20250111 09:01:32.027652 17614 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 186, Committed index: 186, Last appended: 58.187, Last appended by leader: 187, Current term: 58, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:32.027797 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 58 FOLLOWER]: Advancing to term 59
I20250111 09:01:32.028347 17477 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 59 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 59.
I20250111 09:01:32.028501 17338 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 62 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:32.028645 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 59 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:32.028843 17758 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 59 FOLLOWER]: Leader election won for term 59
I20250111 09:01:32.028861 17614 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 59 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 59 because replica has last-logged OpId of term: 58 index: 187, which is greater than that of the candidate, which has last-logged OpId of term: 57 index: 186.
I20250111 09:01:32.028905 17338 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 62 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:32.028934 17758 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 59 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:32.029088 17758 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 186, Committed index: 186, Last appended: 57.186, Last appended by leader: 186, Current term: 59, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:32.029362 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 62 candidate_status { last_received { term: 59 index: 203 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:32.029469 17614 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 61 LEADER]: Stepping down as leader of term 61
I20250111 09:01:32.029515 17614 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 61 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:32.029573 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 62 candidate_status { last_received { term: 59 index: 203 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:32.029655 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 61 FOLLOWER]: Advancing to term 62
I20250111 09:01:32.029631 17614 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 197, Committed index: 197, Last appended: 61.204, Last appended by leader: 204, Current term: 61, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:32.029765 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 61 FOLLOWER]: Advancing to term 62
I20250111 09:01:32.030368 17477 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 62 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 62.
I20250111 09:01:32.030674 17614 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 62 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 62 because replica has last-logged OpId of term: 61 index: 204, which is greater than that of the candidate, which has last-logged OpId of term: 59 index: 203.
I20250111 09:01:32.030666 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 62 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:32.030859 17759 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 62 FOLLOWER]: Leader election won for term 62
I20250111 09:01:32.030942 17759 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 62 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:32.031069 17759 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 203, Committed index: 203, Last appended: 59.203, Last appended by leader: 203, Current term: 62, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:32.031337 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 58 to 59, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 59 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:32.033967 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 61 to 62, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 62 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:32.199595 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:32.200152 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:32.200274 17477 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 59 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:32.200338 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 59 FOLLOWER]: Advancing to term 60
I20250111 09:01:32.201340 17477 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 60 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:32.201655 17477 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 60 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:32.202132 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 60 candidate_status { last_received { term: 57 index: 186 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:32.202131 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 60 candidate_status { last_received { term: 57 index: 186 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:32.202234 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 59 FOLLOWER]: Advancing to term 60
I20250111 09:01:32.202234 17338 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 59 LEADER]: Stepping down as leader of term 59
I20250111 09:01:32.202271 17338 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 59 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:32.202253 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:32.202329 17477 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 62 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:32.202373 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 62 FOLLOWER]: Advancing to term 63
I20250111 09:01:32.202402 17338 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 186, Committed index: 186, Last appended: 59.187, Last appended by leader: 187, Current term: 59, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:32.202529 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 59 FOLLOWER]: Advancing to term 60
I20250111 09:01:32.202929 17477 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 63 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:32.203163 17614 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 60 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 60 because replica has last-logged OpId of term: 58 index: 187, which is greater than that of the candidate, which has last-logged OpId of term: 57 index: 186.
I20250111 09:01:32.203219 17477 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 63 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:32.203495 17338 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 60 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 60 because replica has last-logged OpId of term: 59 index: 187, which is greater than that of the candidate, which has last-logged OpId of term: 57 index: 186.
I20250111 09:01:32.203536 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 63 candidate_status { last_received { term: 59 index: 198 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:32.203644 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 62 FOLLOWER]: Advancing to term 63
I20250111 09:01:32.203796 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 60 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:32.204083 17804 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 60 FOLLOWER]: Leader election lost for term 60. Reason: could not achieve majority
I20250111 09:01:32.204106 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 63 candidate_status { last_received { term: 59 index: 198 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:32.204197 17338 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 62 LEADER]: Stepping down as leader of term 62
I20250111 09:01:32.204249 17338 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 62 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:32.204391 17338 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 203, Committed index: 203, Last appended: 62.204, Last appended by leader: 204, Current term: 62, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:32.204546 17614 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 63 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 63 because replica has last-logged OpId of term: 61 index: 204, which is greater than that of the candidate, which has last-logged OpId of term: 59 index: 198.
I20250111 09:01:32.204551 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 62 FOLLOWER]: Advancing to term 63
I20250111 09:01:32.205344 17338 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 63 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 63 because replica has last-logged OpId of term: 62 index: 204, which is greater than that of the candidate, which has last-logged OpId of term: 59 index: 198.
I20250111 09:01:32.205778 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 63 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:32.205976 17804 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 63 FOLLOWER]: Leader election lost for term 63. Reason: could not achieve majority
I20250111 09:01:32.600730 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:32.601349 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:32.601502 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 60 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:32.601574 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 60 FOLLOWER]: Advancing to term 61
I20250111 09:01:32.602526 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 61 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:32.602814 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 61 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:32.603153 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 61 candidate_status { last_received { term: 58 index: 187 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:32.603168 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 61 candidate_status { last_received { term: 58 index: 187 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:32.603286 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 60 FOLLOWER]: Advancing to term 61
I20250111 09:01:32.603317 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 60 FOLLOWER]: Advancing to term 61
I20250111 09:01:32.603529 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:32.603627 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 63 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:32.603685 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 63 FOLLOWER]: Advancing to term 64
I20250111 09:01:32.604256 17477 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 61 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 61.
I20250111 09:01:32.604385 17338 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 61 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 61 because replica has last-logged OpId of term: 59 index: 187, which is greater than that of the candidate, which has last-logged OpId of term: 58 index: 187.
I20250111 09:01:32.604547 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 61 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:32.604595 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 64 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:32.604753 17789 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 61 FOLLOWER]: Leader election won for term 61
I20250111 09:01:32.604909 17789 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 61 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:32.604938 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 64 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:32.605096 17789 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 186, Committed index: 186, Last appended: 58.187, Last appended by leader: 187, Current term: 61, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:32.605212 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 64 candidate_status { last_received { term: 61 index: 204 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:32.605309 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 63 FOLLOWER]: Advancing to term 64
I20250111 09:01:32.605345 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 64 candidate_status { last_received { term: 61 index: 204 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:32.605438 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 63 FOLLOWER]: Advancing to term 64
I20250111 09:01:32.605994 17338 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 64 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 64 because replica has last-logged OpId of term: 62 index: 204, which is greater than that of the candidate, which has last-logged OpId of term: 61 index: 204.
I20250111 09:01:32.606189 17477 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 64 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 64.
I20250111 09:01:32.606398 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 64 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:32.606775 17789 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 64 FOLLOWER]: Leader election won for term 64
I20250111 09:01:32.606850 17789 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 64 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:32.606957 17789 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 197, Committed index: 197, Last appended: 61.204, Last appended by leader: 204, Current term: 64, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:32.608345 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 59 to 61, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 61 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:32.608548 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 62 to 64, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 64 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:32.952462 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:32.953069 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:32.953197 17338 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 61 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:32.953263 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 61 FOLLOWER]: Advancing to term 62
I20250111 09:01:32.954063 17338 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 62 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:32.954383 17338 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 62 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:32.954681 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 62 candidate_status { last_received { term: 59 index: 187 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:32.954735 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 62 candidate_status { last_received { term: 59 index: 187 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:32.954813 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 61 FOLLOWER]: Advancing to term 62
I20250111 09:01:32.954880 17614 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 61 LEADER]: Stepping down as leader of term 61
I20250111 09:01:32.954931 17614 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 61 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:32.955123 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:32.955097 17614 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 186, Committed index: 186, Last appended: 61.188, Last appended by leader: 188, Current term: 61, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:32.955211 17338 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 64 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:32.955236 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 61 FOLLOWER]: Advancing to term 62
I20250111 09:01:32.955261 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 64 FOLLOWER]: Advancing to term 65
I20250111 09:01:32.955781 17477 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 62 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 62.
I20250111 09:01:32.955999 17338 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 65 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:32.956050 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 62 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:32.956192 17614 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 62 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 62 because replica has last-logged OpId of term: 61 index: 188, which is greater than that of the candidate, which has last-logged OpId of term: 59 index: 187.
I20250111 09:01:32.956300 17338 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 65 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:32.956352 17809 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 62 FOLLOWER]: Leader election won for term 62
I20250111 09:01:32.956482 17809 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 62 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:32.956589 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 65 candidate_status { last_received { term: 62 index: 204 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:32.956605 17809 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 186, Committed index: 186, Last appended: 59.187, Last appended by leader: 187, Current term: 62, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:32.956670 17614 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 64 LEADER]: Stepping down as leader of term 64
I20250111 09:01:32.956709 17614 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 64 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:32.956719 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 65 candidate_status { last_received { term: 62 index: 204 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:32.956801 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 64 FOLLOWER]: Advancing to term 65
I20250111 09:01:32.956831 17614 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 197, Committed index: 197, Last appended: 64.205, Last appended by leader: 205, Current term: 64, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:32.956952 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 64 FOLLOWER]: Advancing to term 65
I20250111 09:01:32.957898 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 61 to 62, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 62 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:32.958979 17477 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 65 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 65.
I20250111 09:01:32.959025 17614 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 65 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 65 because replica has last-logged OpId of term: 64 index: 205, which is greater than that of the candidate, which has last-logged OpId of term: 62 index: 204.
I20250111 09:01:32.959376 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 65 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters: b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:32.959565 17810 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 65 FOLLOWER]: Leader election won for term 65
I20250111 09:01:32.959638 17810 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 65 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:32.959739 17810 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 203, Committed index: 203, Last appended: 62.204, Last appended by leader: 204, Current term: 65, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:32.960800 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 64 to 65, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 65 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:33.345546 17614 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 65 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 64 index: 205. Preceding OpId from leader: term: 65 index: 205. (term mismatch)
I20250111 09:01:33.345638 17614 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 204
I20250111 09:01:33.345662 17614 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted NO_OP operation due to leader change: 64.205
I20250111 09:01:33.345692 17614 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 65 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:33.346117 17810 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 205, Last known committed idx: 197, Time since last communication: 0.000s
I20250111 09:01:33.346230 17810 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 61.204 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 197
I20250111 09:01:33.346833 17614 raft_consensus.cc:1212] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 65 FOLLOWER]: Deduplicated request from leader. Original: 59.197->[59.198-65.205] Dedup: 59.203->[62.204-65.205]
I20250111 09:01:33.346904 17614 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 203
I20250111 09:01:33.346928 17614 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted NO_OP operation due to leader change: 61.204
I20250111 09:01:33.346958 17614 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 65 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:33.349188 17477 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 65 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 59 index: 198. Preceding OpId from leader: term: 65 index: 205. (index mismatch)
I20250111 09:01:33.349687 17809 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 205, Last known committed idx: 196, Time since last communication: 0.000s
W20250111 09:01:33.350128 17754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:33.350260 17754 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=216, attempt_no=6}) took 1621 ms. Trace:
I20250111 09:01:33.350328 17754 rpcz_store.cc:276] 0111 09:01:31.728685 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:31.728728 (+ 43us) service_pool.cc:225] Handling call
0111 09:01:33.350251 (+1621523us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:31.729609 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:33.349956 (+1620347us) op_driver.cc:389] HandleFailure(Illegal state: Transaction 24 already open)
0111 09:01:33.349964 (+ 8us) participant_op.cc:94] Released txn lock
0111 09:01:33.349971 (+ 7us) participant_op.cc:287] FINISH: Op aborted
Related trace 'op':
0111 09:01:33.350109 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER)
0111 09:01:33.350113 (+ 4us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:33.350116 (+ 3us) write_op.cc:454] Released schema lock
0111 09:01:33.350118 (+ 2us) write_op.cc:335] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":40,"txn-participant-registration.run_cpu_time_us":263,"txn-participant-registration.run_wall_time_us":686,"child_traces":[["op",{"prepare.queue_time_us":28,"child_traces":[["op",{}]]}]]}
W20250111 09:01:33.351615 17754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:33.351745 17754 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=224, attempt_no=5}) took 1620 ms. Trace:
I20250111 09:01:33.351790 17754 rpcz_store.cc:276] 0111 09:01:31.731015 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:31.731060 (+ 45us) service_pool.cc:225] Handling call
0111 09:01:33.351737 (+1620677us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:33.351459 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:33.351484 (+ 25us) op_driver.cc:389] HandleFailure(Illegal state: Transaction 28 already open)
0111 09:01:33.351488 (+ 4us) participant_op.cc:94] Released txn lock
0111 09:01:33.351491 (+ 3us) participant_op.cc:287] FINISH: Op aborted
Related trace 'op':
0111 09:01:33.351604 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER)
0111 09:01:33.351607 (+ 3us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:33.351609 (+ 2us) write_op.cc:454] Released schema lock
0111 09:01:33.351610 (+ 1us) write_op.cc:335] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":39,"txn-participant-registration.run_cpu_time_us":301,"txn-participant-registration.run_wall_time_us":725,"child_traces":[["op",{"prepare.queue_time_us":1619516,"child_traces":[["op",{}]]}]]}
W20250111 09:01:33.352459 17754 tablet_replica.cc:1307] Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.003s ago.: cancelling pending write operations [suppressed 22 similar messages]
W20250111 09:01:33.352519 17754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.003s ago.
I20250111 09:01:33.352615 17754 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=226, attempt_no=4}) took 1620 ms. Trace:
I20250111 09:01:33.352653 17754 rpcz_store.cc:276] 0111 09:01:31.731871 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:31.731912 (+ 41us) service_pool.cc:225] Handling call
0111 09:01:33.352607 (+1620695us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:33.352295 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:33.352312 (+ 17us) participant_op.cc:199] PREPARE: finished
0111 09:01:33.352330 (+ 18us) op_driver.cc:389] HandleFailure(Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.003s ago.)
0111 09:01:33.352334 (+ 4us) participant_op.cc:94] Released txn lock
0111 09:01:33.352342 (+ 8us) participant_op.cc:287] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":16,"txn-participant-registration.run_cpu_time_us":288,"txn-participant-registration.run_wall_time_us":925,"child_traces":[["op",{"prepare.queue_time_us":1619319}]]}
W20250111 09:01:33.353225 17754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:33.353339 17754 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=223, attempt_no=5}) took 1621 ms. Trace:
I20250111 09:01:33.353379 17754 rpcz_store.cc:276] 0111 09:01:31.732135 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:31.732170 (+ 35us) service_pool.cc:225] Handling call
0111 09:01:33.353331 (+1621161us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:33.353100 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:33.353118 (+ 18us) op_driver.cc:389] HandleFailure(Illegal state: Transaction 27 already open)
0111 09:01:33.353122 (+ 4us) participant_op.cc:94] Released txn lock
0111 09:01:33.353124 (+ 2us) participant_op.cc:287] FINISH: Op aborted
Related trace 'op':
0111 09:01:33.353213 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER)
0111 09:01:33.353217 (+ 4us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:33.353219 (+ 2us) write_op.cc:454] Released schema lock
0111 09:01:33.353221 (+ 2us) write_op.cc:335] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":42,"txn-participant-registration.run_cpu_time_us":319,"txn-participant-registration.run_wall_time_us":716,"child_traces":[["op",{"prepare.queue_time_us":1620050,"child_traces":[["op",{}]]}]]}
W20250111 09:01:33.354038 17754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:33.354163 17754 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=214, attempt_no=7}) took 1622 ms. Trace:
I20250111 09:01:33.354209 17754 rpcz_store.cc:276] 0111 09:01:31.732081 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:31.732109 (+ 28us) service_pool.cc:225] Handling call
0111 09:01:33.354154 (+1622045us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:33.353891 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:33.353913 (+ 22us) op_driver.cc:389] HandleFailure(Illegal state: Transaction 25 already open)
0111 09:01:33.353917 (+ 4us) participant_op.cc:94] Released txn lock
0111 09:01:33.353920 (+ 3us) participant_op.cc:287] FINISH: Op aborted
Related trace 'op':
0111 09:01:33.354021 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER)
0111 09:01:33.354025 (+ 4us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:33.354029 (+ 4us) write_op.cc:454] Released schema lock
0111 09:01:33.354032 (+ 3us) write_op.cc:335] FINISH: Op aborted
Metrics: {"spinlock_wait_cycles":7552,"txn-participant-registration.queue_time_us":18,"txn-participant-registration.run_cpu_time_us":336,"txn-participant-registration.run_wall_time_us":847,"child_traces":[["op",{"mutex_wait_us":29,"prepare.queue_time_us":1620840,"child_traces":[["op",{}]]}]]}
W20250111 09:01:33.354739 17754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.005s ago.
I20250111 09:01:33.354862 17754 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=222, attempt_no=5}) took 1621 ms. Trace:
I20250111 09:01:33.354902 17754 rpcz_store.cc:276] 0111 09:01:31.733317 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:31.733358 (+ 41us) service_pool.cc:225] Handling call
0111 09:01:33.354854 (+1621496us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:33.354692 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:33.354706 (+ 14us) participant_op.cc:199] PREPARE: finished
0111 09:01:33.354721 (+ 15us) op_driver.cc:389] HandleFailure(Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.005s ago.)
0111 09:01:33.354724 (+ 3us) participant_op.cc:94] Released txn lock
0111 09:01:33.354729 (+ 5us) participant_op.cc:287] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":36,"txn-participant-registration.run_cpu_time_us":304,"txn-participant-registration.run_wall_time_us":796,"child_traces":[["op",{"prepare.queue_time_us":1620400}]]}
W20250111 09:01:33.355216 17754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.006s ago.
I20250111 09:01:33.355320 17754 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=213, attempt_no=6}) took 1620 ms. Trace:
I20250111 09:01:33.355365 17754 rpcz_store.cc:276] 0111 09:01:31.735336 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:31.735380 (+ 44us) service_pool.cc:225] Handling call
0111 09:01:33.355313 (+1619933us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:33.355175 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:33.355187 (+ 12us) participant_op.cc:199] PREPARE: finished
0111 09:01:33.355200 (+ 13us) op_driver.cc:389] HandleFailure(Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.006s ago.)
0111 09:01:33.355203 (+ 3us) participant_op.cc:94] Released txn lock
0111 09:01:33.355207 (+ 4us) participant_op.cc:287] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":38,"txn-participant-registration.run_cpu_time_us":258,"txn-participant-registration.run_wall_time_us":699,"child_traces":[["op",{"prepare.queue_time_us":1618943}]]}
W20250111 09:01:33.355638 17754 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.006s ago.
I20250111 09:01:33.355729 17754 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.ParticipateInTransaction from 127.14.61.3:36423 (ReqId={client: c602c61032fa436092728430b0406df9, seq_no=163, attempt_no=8}) took 1616 ms. Trace:
I20250111 09:01:33.355767 17754 rpcz_store.cc:276] 0111 09:01:31.739078 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:31.739107 (+ 29us) service_pool.cc:225] Handling call
0111 09:01:33.355721 (+1616614us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:33.355605 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:33.355615 (+ 10us) participant_op.cc:199] PREPARE: finished
0111 09:01:33.355629 (+ 14us) op_driver.cc:389] HandleFailure(Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.006s ago.)
0111 09:01:33.355632 (+ 3us) participant_op.cc:94] Released txn lock
0111 09:01:33.355634 (+ 2us) participant_op.cc:287] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"prepare.queue_time_us":1616382}]]}
W20250111 09:01:33.356087 17754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:33.356191 17754 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=225, attempt_no=5}) took 1617 ms. Trace:
I20250111 09:01:33.356230 17754 rpcz_store.cc:276] 0111 09:01:31.738495 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:31.738535 (+ 40us) service_pool.cc:225] Handling call
0111 09:01:33.356184 (+1617649us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:33.355959 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:33.355979 (+ 20us) op_driver.cc:389] HandleFailure(Illegal state: Transaction 26 already open)
0111 09:01:33.355984 (+ 5us) participant_op.cc:94] Released txn lock
0111 09:01:33.355986 (+ 2us) participant_op.cc:287] FINISH: Op aborted
Related trace 'op':
0111 09:01:33.356076 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER)
0111 09:01:33.356079 (+ 3us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:33.356082 (+ 3us) write_op.cc:454] Released schema lock
0111 09:01:33.356084 (+ 2us) write_op.cc:335] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":33,"txn-participant-registration.run_cpu_time_us":258,"txn-participant-registration.run_wall_time_us":728,"child_traces":[["op",{"prepare.queue_time_us":1616557,"child_traces":[["op",{}]]}]]}
W20250111 09:01:33.363562 17825 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 32 similar messages]
W20250111 09:01:33.363664 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.371939 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.376277 17815 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 33 similar messages]
W20250111 09:01:33.377987 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.380198 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.381362 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:33.383634 17614 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 62 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 61 index: 188. Preceding OpId from leader: term: 62 index: 189. (index mismatch)
I20250111 09:01:33.385041 17477 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 62 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 57 index: 186. Preceding OpId from leader: term: 62 index: 189. (index mismatch)
I20250111 09:01:33.385936 17809 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 188, Last known committed idx: 186, Time since last communication: 0.000s
I20250111 09:01:33.386034 17810 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 188, Last known committed idx: 186, Time since last communication: 0.000s
I20250111 09:01:33.386096 17810 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 61.188 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 186
I20250111 09:01:33.387142 17611 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 186
I20250111 09:01:33.387198 17611 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted NO_OP operation due to leader change: 58.187
I20250111 09:01:33.387228 17611 raft_consensus.cc:2883] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 62 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:01:33.387269 17611 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted NO_OP operation due to leader change: 61.188
I20250111 09:01:33.387301 17611 raft_consensus.cc:2883] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 62 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20250111 09:01:33.389441 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:33.445762 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:33.446288 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:33.446960 17477 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 62 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:33.447041 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 62 FOLLOWER]: Advancing to term 63
I20250111 09:01:33.448084 17477 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 63 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:33.449725 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 63 candidate_status { last_received { term: 62 index: 201 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:33.449839 17338 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 62 LEADER]: Stepping down as leader of term 62
I20250111 09:01:33.449878 17338 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 62 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:33.449985 17338 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 201, Committed index: 201, Last appended: 62.205, Last appended by leader: 205, Current term: 62, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:33.450074 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 62 FOLLOWER]: Advancing to term 63
I20250111 09:01:33.451032 17338 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 63 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 63 because replica has last-logged OpId of term: 62 index: 205, which is greater than that of the candidate, which has last-logged OpId of term: 62 index: 201.
W20250111 09:01:33.451781 17845 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.453374 17826 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:33.454169 17474 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 63 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 62. Current term is 63. Ops: [62.202-62.203]
I20250111 09:01:33.455958 17477 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 63 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
W20250111 09:01:33.457080 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:33.457175 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:33.457206 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 63 candidate_status { last_received { term: 62 index: 201 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:33.457260 17477 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 65 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:33.457306 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 65 FOLLOWER]: Advancing to term 66
I20250111 09:01:33.457307 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 62 FOLLOWER]: Advancing to term 63
I20250111 09:01:33.458015 17611 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 63 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 63 because replica has last-logged OpId of term: 62 index: 202, which is greater than that of the candidate, which has last-logged OpId of term: 62 index: 201.
I20250111 09:01:33.458068 17477 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 66 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:33.458425 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 63 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:33.458494 17474 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 66 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 65. Current term is 66. Ops: [65.235-65.236]
I20250111 09:01:33.458523 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 66 candidate_status { last_received { term: 65 index: 234 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:33.458608 17338 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 65 LEADER]: Stepping down as leader of term 65
I20250111 09:01:33.458645 17338 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 65 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:33.458710 17822 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 63 FOLLOWER]: Leader election lost for term 63. Reason: could not achieve majority
I20250111 09:01:33.458745 17338 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 234, Committed index: 234, Last appended: 65.236, Last appended by leader: 236, Current term: 65, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:33.458899 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 65 FOLLOWER]: Advancing to term 66
W20250111 09:01:33.458962 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.459648 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:33.459944 17338 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 66 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 66 because replica has last-logged OpId of term: 65 index: 236, which is greater than that of the candidate, which has last-logged OpId of term: 65 index: 234.
I20250111 09:01:33.460496 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 66 candidate_status { last_received { term: 65 index: 234 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:33.460597 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 65 FOLLOWER]: Advancing to term 66
I20250111 09:01:33.461045 17477 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 66 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:33.461362 17611 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 66 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 66 because replica has last-logged OpId of term: 65 index: 235, which is greater than that of the candidate, which has last-logged OpId of term: 65 index: 234.
I20250111 09:01:33.461580 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 66 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:33.461721 17822 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 66 FOLLOWER]: Leader election lost for term 66. Reason: could not achieve majority
W20250111 09:01:33.465552 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.466217 17852 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.467620 17852 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.471011 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.473389 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.476727 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.476802 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.480026 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.485692 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.486817 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.487691 17852 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.497412 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.497447 17852 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.500636 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.508055 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.508121 17826 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.511726 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.518903 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.518950 17829 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.525564 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.533350 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.533429 17852 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.538878 17826 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.547739 17826 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.547783 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.552392 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.562697 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.562765 17829 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.567339 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.579006 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.580119 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.585747 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.595854 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.597682 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.602299 17829 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.614631 17829 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.614738 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.621309 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.631000 17852 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.631052 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.641741 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.650753 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.652850 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.660620 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.671623 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.673775 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.679507 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.691191 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.694819 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.702577 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.713949 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.717641 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.727473 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.737721 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.739694 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.751492 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.760219 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.766021 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.775956 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.787917 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.793136 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.800046 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.815318 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.821327 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.829257 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.842003 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.850704 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.857542 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.873557 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.878727 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.887588 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.901814 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.905934 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.919821 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.933504 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.938052 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:33.951684 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.964788 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.968921 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.984082 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:33.995177 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.002140 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.019001 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.029912 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.035691 17851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:34.051424 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:34.051864 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:34.051973 17611 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 63 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:34.052045 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 63 FOLLOWER]: Advancing to term 64
W20250111 09:01:34.052469 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:34.053120 17611 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 64 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:34.053429 17611 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 64 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:34.053678 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 64 candidate_status { last_received { term: 62 index: 202 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:34.053778 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 63 FOLLOWER]: Advancing to term 64
I20250111 09:01:34.053836 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 64 candidate_status { last_received { term: 62 index: 202 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:34.053977 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 63 FOLLOWER]: Advancing to term 64
I20250111 09:01:34.054689 17338 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 64 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 64 because replica has last-logged OpId of term: 62 index: 205, which is greater than that of the candidate, which has last-logged OpId of term: 62 index: 202.
I20250111 09:01:34.054716 17477 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 64 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 64.
I20250111 09:01:34.054983 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 64 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:34.055174 17847 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 64 FOLLOWER]: Leader election won for term 64
I20250111 09:01:34.055322 17847 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 64 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:34.055431 17847 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 201, Committed index: 201, Last appended: 62.202, Last appended by leader: 202, Current term: 64, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:34.055665 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:34.055761 17611 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 66 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:34.055824 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 66 FOLLOWER]: Advancing to term 67
I20250111 09:01:34.056597 17611 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 67 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:34.056912 17611 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 67 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:34.057161 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 67 candidate_status { last_received { term: 65 index: 235 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:34.057204 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 67 candidate_status { last_received { term: 65 index: 235 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:34.057263 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 66 FOLLOWER]: Advancing to term 67
I20250111 09:01:34.057292 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 66 FOLLOWER]: Advancing to term 67
I20250111 09:01:34.057901 17338 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 67 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 67 because replica has last-logged OpId of term: 65 index: 236, which is greater than that of the candidate, which has last-logged OpId of term: 65 index: 235.
I20250111 09:01:34.057976 17477 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 67 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 67.
I20250111 09:01:34.058215 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 67 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:34.058120 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 62 to 64, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 64 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:34.058416 17847 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 67 FOLLOWER]: Leader election won for term 67
I20250111 09:01:34.058519 17847 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 67 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:34.058617 17847 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 232, Committed index: 232, Last appended: 65.235, Last appended by leader: 235, Current term: 67, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:34.061486 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 65 to 67, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 67 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:34.064626 17477 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 64 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 62 index: 201. Preceding OpId from leader: term: 64 index: 204. (index mismatch)
I20250111 09:01:34.064708 17338 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 64 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 62 index: 205. Preceding OpId from leader: term: 64 index: 204. (term mismatch)
I20250111 09:01:34.064774 17338 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 203
I20250111 09:01:34.064812 17338 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 62.204
W20250111 09:01:34.064898 17338 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Aborted: Op aborted by new leader
I20250111 09:01:34.064990 17853 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 203, Last known committed idx: 199, Time since last communication: 0.000s
I20250111 09:01:34.065107 17338 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 62.205
W20250111 09:01:34.065207 17338 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Aborted: Op aborted by new leader
I20250111 09:01:34.065959 17853 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 203, Last known committed idx: 201, Time since last communication: 0.000s
I20250111 09:01:34.066052 17853 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer fb270e42d7c9400f8c05bb9565db1972 log is divergent from this leader: its last log entry 62.203 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 201
I20250111 09:01:34.066576 17338 raft_consensus.cc:1212] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 64 FOLLOWER]: Deduplicated request from leader. Original: 62.201->[62.202-64.204] Dedup: 62.202->[64.203-64.204]
I20250111 09:01:34.066659 17338 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 202
I20250111 09:01:34.066686 17338 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted WRITE_OP operation due to leader change: 62.203
W20250111 09:01:34.066771 17338 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Aborted: Op aborted by new leader
W20250111 09:01:34.068769 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:34.070607 17862 mvcc.cc:204] Tried to move back new op lower bound from 7113056641285570560 to 7113056641252245504. Current Snapshot: MvccSnapshot[applied={T|T < 7113056638678372352 or (T in {7113056638745292800})}]
I20250111 09:01:34.070787 17866 mvcc.cc:204] Tried to move back new op lower bound from 7113056641285570560 to 7113056641252245504. Current Snapshot: MvccSnapshot[applied={T|T < 7113056638745292800}]
W20250111 09:01:34.071214 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:34.080252 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:34.080767 17339 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:34.080870 17339 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 64 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:34.080924 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 64 FOLLOWER]: Advancing to term 65
I20250111 09:01:34.082053 17339 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 65 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:34.082404 17339 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 65 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:34.082406 17338 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 65 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 64. Current term is 65. Ops: [64.208-64.209]
I20250111 09:01:34.082784 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 65 candidate_status { last_received { term: 64 index: 207 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:34.082887 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 64 FOLLOWER]: Advancing to term 65
I20250111 09:01:34.083736 17476 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 67 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 65 index: 234. Preceding OpId from leader: term: 67 index: 237. (index mismatch)
I20250111 09:01:34.083793 17474 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 65 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 65 because replica has last-logged OpId of term: 64 index: 209, which is greater than that of the candidate, which has last-logged OpId of term: 64 index: 207.
I20250111 09:01:34.083799 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 65 candidate_status { last_received { term: 64 index: 207 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:34.083912 17611 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 64 LEADER]: Stepping down as leader of term 64
I20250111 09:01:34.083952 17611 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 64 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:34.084061 17611 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 209, Committed index: 209, Last appended: 64.211, Last appended by leader: 211, Current term: 64, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:34.084179 17339 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 67 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 65 index: 236. Preceding OpId from leader: term: 67 index: 237. (index mismatch)
I20250111 09:01:34.084167 17867 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: INVALID_TERM, Last received: 64.207, Next index: 208, Last known committed idx: 205, Time since last communication: 0.000s
I20250111 09:01:34.084378 17867 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 236, Last known committed idx: 232, Time since last communication: 0.000s
I20250111 09:01:34.084537 17853 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 236, Last known committed idx: 234, Time since last communication: 0.000s
I20250111 09:01:34.084615 17853 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer fb270e42d7c9400f8c05bb9565db1972 log is divergent from this leader: its last log entry 65.236 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 234
I20250111 09:01:34.084771 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 64 FOLLOWER]: Advancing to term 65
W20250111 09:01:34.084618 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:34.085052 17477 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 65 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 64. Current term is 65. Ops: [64.210-64.210]
I20250111 09:01:34.085727 17611 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 65 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 65 because replica has last-logged OpId of term: 64 index: 211, which is greater than that of the candidate, which has last-logged OpId of term: 64 index: 207.
I20250111 09:01:34.086014 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 65 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:34.086195 17864 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 65 FOLLOWER]: Leader election lost for term 65. Reason: could not achieve majority
I20250111 09:01:34.086773 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:34.086874 17340 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 67 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:34.086937 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 67 FOLLOWER]: Advancing to term 68
W20250111 09:01:34.087690 17852 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:34.087814 17340 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 68 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:34.088117 17340 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 68 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:34.090416 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 68 candidate_status { last_received { term: 65 index: 236 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:34.090544 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 67 FOLLOWER]: Advancing to term 68
I20250111 09:01:34.091441 17477 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 68 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 68 because replica has last-logged OpId of term: 67 index: 238, which is greater than that of the candidate, which has last-logged OpId of term: 65 index: 236.
I20250111 09:01:34.091724 17339 raft_consensus.cc:1212] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 68 FOLLOWER]: Deduplicated request from leader. Original: 65.234->[65.235-67.237] Dedup: 65.235->[67.236-67.237]
I20250111 09:01:34.091801 17339 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 68 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 67. Current term is 68. Ops: [65.235-67.237]
I20250111 09:01:34.092044 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 68 candidate_status { last_received { term: 65 index: 236 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:34.092140 17611 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 67 LEADER]: Stepping down as leader of term 67
I20250111 09:01:34.092182 17611 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 67 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
W20250111 09:01:34.092195 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:34.092301 17611 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 238, Committed index: 238, Last appended: 67.239, Last appended by leader: 239, Current term: 67, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:34.092396 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 67 FOLLOWER]: Advancing to term 68
W20250111 09:01:34.092491 17863 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:34.093266 17611 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 68 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 68 because replica has last-logged OpId of term: 67 index: 239, which is greater than that of the candidate, which has last-logged OpId of term: 65 index: 236.
I20250111 09:01:34.093590 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 68 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:34.093570 17477 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 68 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 67. Current term is 68. Ops: [67.239-67.239]
I20250111 09:01:34.093879 17864 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 68 FOLLOWER]: Leader election lost for term 68. Reason: could not achieve majority
W20250111 09:01:34.095984 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.097193 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.101619 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.105687 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.111405 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.114531 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.120579 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.123149 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.124619 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.128813 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.134879 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.139638 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.146759 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.151891 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.159792 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.160501 17852 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.167153 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.170154 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.178975 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.183283 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.194491 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.196332 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.196453 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.209332 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.213351 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.225167 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.230111 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.237789 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.241513 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.249138 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.258503 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.265529 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.278321 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.280534 17852 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.284181 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.298288 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.304070 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.322425 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.322448 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.325475 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.346426 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.348424 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.364594 17825 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 49 similar messages]
W20250111 09:01:34.364702 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.370599 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.371989 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.396777 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.397153 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.406935 17852 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 39 similar messages]
W20250111 09:01:34.406998 17852 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.422986 17844 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 56 similar messages]
W20250111 09:01:34.423040 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.423950 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.450673 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.451995 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.452029 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.479730 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.482347 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.497366 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.507140 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.513950 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.535830 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.545086 17852 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.545102 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.567847 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.578254 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.590024 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.600209 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:34.608891 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.631174 17844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.638108 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:34.639983 17825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:34.657509 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:34.657929 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:34.658031 17477 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 65 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:34.658102 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 65 FOLLOWER]: Advancing to term 66
I20250111 09:01:34.658803 17477 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 66 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:34.659116 17477 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 66 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:34.659375 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 66 candidate_status { last_received { term: 64 index: 209 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:34.659375 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 66 candidate_status { last_received { term: 64 index: 209 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:34.659492 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 65 FOLLOWER]: Advancing to term 66
I20250111 09:01:34.659492 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 65 FOLLOWER]: Advancing to term 66
I20250111 09:01:34.659696 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:34.659770 17477 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 68 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:34.659804 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 68 FOLLOWER]: Advancing to term 69
I20250111 09:01:34.660354 17339 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 66 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 66.
I20250111 09:01:34.660403 17611 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 66 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 66 because replica has last-logged OpId of term: 64 index: 211, which is greater than that of the candidate, which has last-logged OpId of term: 64 index: 209.
I20250111 09:01:34.660430 17477 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 69 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:34.660674 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 66 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters: b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:34.660732 17477 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 69 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:34.660921 17881 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 66 FOLLOWER]: Leader election won for term 66
I20250111 09:01:34.661088 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 69 candidate_status { last_received { term: 67 index: 238 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:34.661116 17881 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 66 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:34.661187 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 68 FOLLOWER]: Advancing to term 69
I20250111 09:01:34.661186 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 69 candidate_status { last_received { term: 67 index: 238 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:34.661279 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 68 FOLLOWER]: Advancing to term 69
I20250111 09:01:34.661268 17881 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 208, Committed index: 208, Last appended: 64.209, Last appended by leader: 208, Current term: 66, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:34.661947 17611 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 69 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 69 because replica has last-logged OpId of term: 67 index: 239, which is greater than that of the candidate, which has last-logged OpId of term: 67 index: 238.
I20250111 09:01:34.662061 17339 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 69 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 69.
I20250111 09:01:34.662267 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 69 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters: b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:34.662573 17882 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 69 FOLLOWER]: Leader election won for term 69
I20250111 09:01:34.662537 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 64 to 66, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 66 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:34.662689 17882 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 69 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:34.662798 17882 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 237, Committed index: 237, Last appended: 67.238, Last appended by leader: 238, Current term: 69, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:34.665544 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 67 to 69, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 69 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:35.059245 17611 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 66 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 64 index: 211. Preceding OpId from leader: term: 66 index: 210. (term mismatch)
I20250111 09:01:35.059327 17611 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 209
I20250111 09:01:35.059347 17611 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 64.210
W20250111 09:01:35.059459 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:01:35.059671 17611 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 64.211
W20250111 09:01:35.059756 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:01:35.060196 17882 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 210, Last known committed idx: 209, Time since last communication: 0.000s
I20250111 09:01:35.062470 17339 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 66 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 64 index: 207. Preceding OpId from leader: term: 66 index: 210. (index mismatch)
I20250111 09:01:35.062988 17881 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 210, Last known committed idx: 205, Time since last communication: 0.000s
I20250111 09:01:35.102066 17611 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 69 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 67 index: 239. Preceding OpId from leader: term: 69 index: 239. (term mismatch)
I20250111 09:01:35.102138 17611 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 238
I20250111 09:01:35.102160 17611 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 67.239
W20250111 09:01:35.102247 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:01:35.102348 17611 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=256, attempt_no=0}) took 1029 ms. Trace:
I20250111 09:01:35.102396 17611 rpcz_store.cc:276] 0111 09:01:34.072993 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:34.073053 (+ 60us) service_pool.cc:225] Handling call
0111 09:01:35.102342 (+1029289us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:34.081888 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:34.081903 (+ 15us) participant_op.cc:199] PREPARE: finished
0111 09:01:34.081919 (+ 16us) participant_op.cc:212] START. Timestamp: 1736586094081885
0111 09:01:34.081920 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:01:34.081985 (+ 65us) log.cc:844] Serialized 66 byte log entry
0111 09:01:34.088359 (+ 6374us) op_driver.cc:464] REPLICATION: finished
0111 09:01:34.088842 (+ 483us) participant_op.cc:265] APPLY: starting
0111 09:01:34.088972 (+ 130us) participant_op.cc:270] APPLY: finished
0111 09:01:34.088989 (+ 17us) log.cc:844] Serialized 15 byte log entry
0111 09:01:34.089631 (+ 642us) participant_op.cc:94] Released txn lock
0111 09:01:34.089636 (+ 5us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:01:34.089789 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 6ab63c6543ad4a0ab83e76267a725edd
0111 09:01:34.089862 (+ 73us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:34.089864 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:01:34.089865 (+ 1us) tablet.cc:659] Decoding operations
0111 09:01:34.089901 (+ 36us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:01:34.089926 (+ 25us) tablet.cc:698] Row locks acquired
0111 09:01:34.089927 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:01:34.089950 (+ 23us) write_op.cc:270] Start()
0111 09:01:34.089964 (+ 14us) write_op.cc:276] Timestamp: P: 1736586094089917 usec, L: 0
0111 09:01:34.089966 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:01:34.091331 (+ 1365us) log.cc:844] Serialized 305 byte log entry
0111 09:01:35.102221 (+1010890us) op_driver.cc:464] REPLICATION: finished
0111 09:01:35.102226 (+ 5us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0111 09:01:35.102230 (+ 4us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:35.102239 (+ 9us) write_op.cc:454] Released schema lock
0111 09:01:35.102243 (+ 4us) write_op.cc:335] FINISH: Op aborted
Metrics: {"thread_start_us":74,"threads_started":1,"txn-participant-registration.queue_time_us":143,"txn-participant-registration.run_cpu_time_us":346,"txn-participant-registration.run_wall_time_us":9482,"child_traces":[["op",{"apply.queue_time_us":448,"apply.run_cpu_time_us":342,"apply.run_wall_time_us":940,"prepare.queue_time_us":603,"prepare.run_cpu_time_us":152,"prepare.run_wall_time_us":152,"raft.queue_time_us":910,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":152,"replication_time_us":6423,"thread_start_us":70,"threads_started":1,"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":25,"prepare.run_cpu_time_us":378,"prepare.run_wall_time_us":2076,"replication_time_us":1012225}]]}]]}
I20250111 09:01:35.103032 17882 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 239, Last known committed idx: 238, Time since last communication: 0.000s
I20250111 09:01:35.105038 17339 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 69 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 65 index: 236. Preceding OpId from leader: term: 69 index: 239. (index mismatch)
I20250111 09:01:35.105952 17882 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 239, Last known committed idx: 234, Time since last communication: 0.000s
I20250111 09:01:35.106045 17882 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer fb270e42d7c9400f8c05bb9565db1972 log is divergent from this leader: its last log entry 65.236 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 234
I20250111 09:01:35.106544 17339 raft_consensus.cc:1212] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 69 FOLLOWER]: Deduplicated request from leader. Original: 65.234->[65.235-69.239] Dedup: 65.235->[67.236-69.239]
I20250111 09:01:35.106604 17339 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 235
I20250111 09:01:35.106618 17339 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted WRITE_OP operation due to leader change: 65.236
W20250111 09:01:35.106676 17339 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Aborted: Op aborted by new leader
I20250111 09:01:35.106767 17339 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39814 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=255, attempt_no=0}) took 1652 ms. Trace:
I20250111 09:01:35.106809 17339 rpcz_store.cc:276] 0111 09:01:33.454748 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:33.454810 (+ 62us) service_pool.cc:225] Handling call
0111 09:01:35.106762 (+1651952us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:33.455090 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 6ab63c6543ad4a0ab83e76267a725edd
0111 09:01:33.455142 (+ 52us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:33.455144 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:01:33.455145 (+ 1us) tablet.cc:659] Decoding operations
0111 09:01:33.455179 (+ 34us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:01:33.455203 (+ 24us) tablet.cc:698] Row locks acquired
0111 09:01:33.455204 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:01:33.455238 (+ 34us) write_op.cc:270] Start()
0111 09:01:33.455253 (+ 15us) write_op.cc:276] Timestamp: P: 1736586093455171 usec, L: 0
0111 09:01:33.455254 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:01:33.455336 (+ 82us) log.cc:844] Serialized 305 byte log entry
0111 09:01:35.106661 (+1651325us) op_driver.cc:464] REPLICATION: finished
0111 09:01:35.106665 (+ 4us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0111 09:01:35.106667 (+ 2us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:35.106672 (+ 5us) write_op.cc:454] Released schema lock
0111 09:01:35.106675 (+ 3us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":34,"prepare.run_cpu_time_us":311,"prepare.run_wall_time_us":455,"raft.queue_time_us":181,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":3,"replication_time_us":1651383}]]}
I20250111 09:01:35.108120 17898 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39814 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=253, attempt_no=0}) took 1663 ms. Trace:
I20250111 09:01:35.108183 17898 rpcz_store.cc:276] 0111 09:01:33.444368 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:33.444468 (+ 100us) service_pool.cc:225] Handling call
0111 09:01:35.108110 (+1663642us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:33.448896 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 6ab63c6543ad4a0ab83e76267a725edd
0111 09:01:33.449140 (+ 244us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:33.449142 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:01:33.449143 (+ 1us) tablet.cc:659] Decoding operations
0111 09:01:33.449184 (+ 41us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:01:33.449212 (+ 28us) tablet.cc:698] Row locks acquired
0111 09:01:33.449214 (+ 2us) write_op.cc:260] PREPARE: finished
0111 09:01:33.449234 (+ 20us) write_op.cc:270] Start()
0111 09:01:33.449250 (+ 16us) write_op.cc:276] Timestamp: P: 1736586093449182 usec, L: 0
0111 09:01:33.449253 (+ 3us) op_driver.cc:348] REPLICATION: starting
0111 09:01:33.449334 (+ 81us) log.cc:844] Serialized 305 byte log entry
0111 09:01:35.107572 (+1658238us) op_driver.cc:464] REPLICATION: finished
0111 09:01:35.107777 (+ 205us) write_op.cc:301] APPLY: starting
0111 09:01:35.107789 (+ 12us) tablet.cc:1364] starting BulkCheckPresence
0111 09:01:35.107827 (+ 38us) tablet.cc:1367] finished BulkCheckPresence
0111 09:01:35.107828 (+ 1us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:01:35.107853 (+ 25us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:01:35.107883 (+ 30us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0111 09:01:35.107886 (+ 3us) write_op.cc:312] APPLY: finished
0111 09:01:35.107921 (+ 35us) log.cc:844] Serialized 25 byte log entry
0111 09:01:35.108051 (+ 130us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:35.108060 (+ 9us) write_op.cc:454] Released schema lock
0111 09:01:35.108063 (+ 3us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":176,"num_ops":1,"prepare.queue_time_us":4174,"prepare.run_cpu_time_us":281,"prepare.run_wall_time_us":473,"replication_time_us":1658304,"thread_start_us":85,"threads_started":1}]]}
W20250111 09:01:35.114966 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.115571 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.119807 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:35.281035 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:35.281965 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:35.282140 17611 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 66 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:35.282239 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 66 FOLLOWER]: Advancing to term 67
I20250111 09:01:35.283263 17611 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 67 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:35.284062 17613 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 67 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 66. Current term is 67. Ops: []
I20250111 09:01:35.284279 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 67 candidate_status { last_received { term: 66 index: 263 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:35.284390 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 66 FOLLOWER]: Advancing to term 67
I20250111 09:01:35.284533 17914 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: INVALID_TERM, Last received: 66.263, Next index: 264, Last known committed idx: 262, Time since last communication: 0.000s
I20250111 09:01:35.284698 17914 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 66 LEADER]: Stepping down as leader of term 66
I20250111 09:01:35.284741 17914 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 66 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:35.284871 17914 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 263, Committed index: 263, Last appended: 66.263, Last appended by leader: 263, Current term: 66, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:35.284983 17914 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 66 FOLLOWER]: Advancing to term 67
I20250111 09:01:35.285401 17338 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 67 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 67.
I20250111 09:01:35.285902 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 67 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:35.286435 17900 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 67 FOLLOWER]: Leader election won for term 67
I20250111 09:01:35.286613 17900 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 67 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:35.286727 17900 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 262, Committed index: 262, Last appended: 66.263, Last appended by leader: 263, Current term: 67, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:35.287905 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:35.288105 17611 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 67 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
W20250111 09:01:35.289693 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:35.289883 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:35.290510 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 67 candidate_status { last_received { term: 66 index: 263 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:35.291338 17474 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 67 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 67.
I20250111 09:01:35.296489 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 66 to 67, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 67 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:35.296804 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 69 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:35.296864 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 69 FOLLOWER]: Advancing to term 70
W20250111 09:01:35.296985 17922 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:35.297704 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 70 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:35.298285 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 70 candidate_status { last_received { term: 69 index: 294 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
W20250111 09:01:35.298313 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:35.298391 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 69 FOLLOWER]: Advancing to term 70
I20250111 09:01:35.299373 17340 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 70 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 70 because replica has last-logged OpId of term: 69 index: 297, which is greater than that of the candidate, which has last-logged OpId of term: 69 index: 294.
I20250111 09:01:35.299744 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 70 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:35.300186 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 70 candidate_status { last_received { term: 69 index: 294 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:35.300282 17474 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 69 LEADER]: Stepping down as leader of term 69
I20250111 09:01:35.300324 17474 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 69 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:35.300447 17474 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 294, Committed index: 294, Last appended: 69.297, Last appended by leader: 297, Current term: 69, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:35.300865 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:35.301131 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 69 FOLLOWER]: Advancing to term 70
I20250111 09:01:35.301880 17474 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 70 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 70 because replica has last-logged OpId of term: 69 index: 297, which is greater than that of the candidate, which has last-logged OpId of term: 69 index: 294.
I20250111 09:01:35.302134 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 70 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:35.302273 17921 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 70 FOLLOWER]: Leader election lost for term 70. Reason: could not achieve majority
W20250111 09:01:35.303328 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.306181 17922 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.307094 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:35.307173 17340 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 67 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 66 index: 263. Preceding OpId from leader: term: 67 index: 265. (index mismatch)
W20250111 09:01:35.307474 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:35.307650 17474 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 67 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 66 index: 263. Preceding OpId from leader: term: 67 index: 265. (index mismatch)
I20250111 09:01:35.307572 17923 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 264, Last known committed idx: 262, Time since last communication: 0.000s
I20250111 09:01:35.308550 17921 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 264, Last known committed idx: 263, Time since last communication: 0.000s
W20250111 09:01:35.309389 17926 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.325963 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.327288 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.327397 17922 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.328104 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.330379 17926 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.331136 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.334039 17926 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.334709 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.335309 17926 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.336217 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.337888 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.337888 17932 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.339972 17932 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.341398 17932 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.342074 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.342733 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.346215 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.346410 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.347227 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.350653 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.352073 17926 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.355520 17926 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.355557 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.356506 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.358532 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.359895 17932 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.364545 17932 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.364574 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.366464 17932 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.367504 17932 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.369653 17908 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 35 similar messages]
W20250111 09:01:35.369705 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.375698 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.378002 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.378567 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.379280 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.383088 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.388732 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.390117 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.391436 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.391458 17926 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.396983 17932 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.401211 17932 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.403335 17932 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.406112 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.406148 17932 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.411751 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.417330 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.417474 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.417761 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.419070 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.427721 17925 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 35 similar messages]
W20250111 09:01:35.427764 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.432075 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.433250 17926 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.433696 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.433777 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.444393 17932 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 23 similar messages]
W20250111 09:01:35.444447 17932 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.446875 17932 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.447315 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.447374 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.450052 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.459252 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.461786 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.461941 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.465579 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.465673 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.477438 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.478636 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.480613 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.482933 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.485178 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.496791 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.498003 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.500182 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.502211 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.505589 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.516429 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.517459 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.520308 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.521332 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.526297 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.536342 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.536342 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.539345 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.541489 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.544579 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.556735 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.557590 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.561753 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.562932 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.566056 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.580185 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.580767 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.585556 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.586724 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.586792 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.601647 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.601715 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.606915 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.608353 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.610911 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.623754 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.625445 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.633376 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.634596 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.636328 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.651240 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.652132 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.658706 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.662305 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.663362 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.677227 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.677276 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.684612 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.687935 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.692056 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.705113 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.705178 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.711015 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.715274 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.718266 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.730965 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.731899 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.737598 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.741852 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.744437 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.758373 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.759426 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.764606 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.770599 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.773912 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.788673 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.790666 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.797276 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.800215 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.805419 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:35.810076 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:35.810550 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:35.810655 17340 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 67 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:35.810719 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 67 FOLLOWER]: Advancing to term 68
I20250111 09:01:35.811561 17340 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 68 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:35.811905 17340 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 68 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:35.812199 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 68 candidate_status { last_received { term: 67 index: 276 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:35.812255 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 68 candidate_status { last_received { term: 67 index: 276 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:35.812311 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 67 FOLLOWER]: Advancing to term 68
I20250111 09:01:35.812345 17614 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 67 LEADER]: Stepping down as leader of term 67
I20250111 09:01:35.812386 17614 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 67 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:35.812518 17614 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 276, Committed index: 276, Last appended: 67.276, Last appended by leader: 276, Current term: 67, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:35.812652 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 67 FOLLOWER]: Advancing to term 68
I20250111 09:01:35.812805 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:35.812904 17340 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 70 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:35.812958 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 70 FOLLOWER]: Advancing to term 71
I20250111 09:01:35.813228 17474 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 68 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 68.
I20250111 09:01:35.813506 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 68 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:35.813555 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 68 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 68.
I20250111 09:01:35.813743 17934 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 68 FOLLOWER]: Leader election won for term 68
I20250111 09:01:35.813998 17340 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 71 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:35.814208 17934 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 68 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:35.814342 17340 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 71 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:35.814359 17934 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 276, Committed index: 276, Last appended: 67.276, Last appended by leader: 276, Current term: 68, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:35.814651 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 71 candidate_status { last_received { term: 69 index: 297 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:35.814749 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 70 FOLLOWER]: Advancing to term 71
I20250111 09:01:35.815518 17474 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 71 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 71.
I20250111 09:01:35.815775 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 71 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:35.816093 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 67 to 68, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 68 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:35.816699 17935 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 71 FOLLOWER]: Leader election won for term 71
I20250111 09:01:35.816720 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 71 candidate_status { last_received { term: 69 index: 297 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:35.816781 17935 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 71 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:35.816798 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 70 FOLLOWER]: Advancing to term 71
I20250111 09:01:35.816907 17935 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 293, Committed index: 293, Last appended: 69.297, Last appended by leader: 297, Current term: 71, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:35.817507 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 71 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 71.
I20250111 09:01:35.818863 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 69 to 71, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 71 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:35.820976 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:35.823923 17474 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 71 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 69 index: 297. Preceding OpId from leader: term: 71 index: 299. (index mismatch)
I20250111 09:01:35.823951 17614 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 71 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 69 index: 294. Preceding OpId from leader: term: 71 index: 299. (index mismatch)
I20250111 09:01:35.824357 17934 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 298, Last known committed idx: 292, Time since last communication: 0.000s
I20250111 09:01:35.824458 17935 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 298, Last known committed idx: 294, Time since last communication: 0.000s
W20250111 09:01:35.834981 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.835369 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:35.840166 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:35.845868 17477 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 68 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 67 index: 276. Preceding OpId from leader: term: 68 index: 278. (index mismatch)
I20250111 09:01:35.846186 17614 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 68 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 67 index: 276. Preceding OpId from leader: term: 68 index: 278. (index mismatch)
I20250111 09:01:35.847424 17935 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 277, Last known committed idx: 276, Time since last communication: 0.000s
I20250111 09:01:35.848419 17934 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 277, Last known committed idx: 276, Time since last communication: 0.000s
W20250111 09:01:35.861213 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:35.863309 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:36.009135 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:36.010107 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:36.010241 17477 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 68 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:36.010300 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 68 FOLLOWER]: Advancing to term 69
I20250111 09:01:36.011242 17477 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 69 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:36.011485 17467 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 69 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 68. Current term is 69. Ops: [68.324-68.324]
I20250111 09:01:36.011615 17477 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 69 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:36.011979 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 69 candidate_status { last_received { term: 68 index: 323 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:36.012086 17340 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 68 LEADER]: Stepping down as leader of term 68
I20250111 09:01:36.012136 17340 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 68 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:36.012251 17340 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 324, Committed index: 324, Last appended: 68.326, Last appended by leader: 326, Current term: 68, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:36.012356 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 68 FOLLOWER]: Advancing to term 69
I20250111 09:01:36.013242 17340 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 69 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 69 because replica has last-logged OpId of term: 68 index: 326, which is greater than that of the candidate, which has last-logged OpId of term: 68 index: 323.
W20250111 09:01:36.014951 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:36.015416 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 69 candidate_status { last_received { term: 68 index: 323 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:36.015523 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 68 FOLLOWER]: Advancing to term 69
I20250111 09:01:36.015987 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:36.016085 17467 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 71 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:36.016144 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 71 FOLLOWER]: Advancing to term 72
I20250111 09:01:36.017059 17467 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 72 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:36.017333 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:36.018355 17613 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 69 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 69 because replica has last-logged OpId of term: 68 index: 324, which is greater than that of the candidate, which has last-logged OpId of term: 68 index: 323.
I20250111 09:01:36.018657 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 72 candidate_status { last_received { term: 71 index: 359 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:36.018757 17340 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 71 LEADER]: Stepping down as leader of term 71
I20250111 09:01:36.018806 17340 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 71 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:36.018769 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 69 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:36.019013 17939 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 69 FOLLOWER]: Leader election lost for term 69. Reason: could not achieve majority
W20250111 09:01:36.019474 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.019928 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:36.018939 17340 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 359, Committed index: 359, Last appended: 71.359, Last appended by leader: 359, Current term: 71, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:36.020346 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 71 FOLLOWER]: Advancing to term 72
W20250111 09:01:36.020632 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.020910 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:36.021103 17340 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 72 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 72.
I20250111 09:01:36.021503 17614 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 69 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 68. Current term is 69. Ops: [68.325-68.326]
I20250111 09:01:36.022099 17474 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 72 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 71. Current term is 72. Ops: []
W20250111 09:01:36.022300 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.022593 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:36.022615 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 72 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:36.022763 17939 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 72 FOLLOWER]: Leader election won for term 72
I20250111 09:01:36.022923 17939 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 72 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:36.023042 17939 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 358, Committed index: 358, Last appended: 71.359, Last appended by leader: 359, Current term: 72, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:36.023705 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 72 candidate_status { last_received { term: 71 index: 359 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:36.023802 17467 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 72 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:36.023808 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 71 FOLLOWER]: Advancing to term 72
W20250111 09:01:36.024230 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:36.024736 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 72 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 72.
W20250111 09:01:36.025504 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:36.026043 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 71 to 72, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 72 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:36.027657 17339 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 72 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 71 index: 359. Preceding OpId from leader: term: 72 index: 361. (index mismatch)
I20250111 09:01:36.028343 17939 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 360, Last known committed idx: 359, Time since last communication: 0.000s
W20250111 09:01:36.028971 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.029078 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:36.030320 17614 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 72 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 71 index: 359. Preceding OpId from leader: term: 72 index: 361. (index mismatch)
W20250111 09:01:36.030334 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.030982 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:36.031478 17968 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 360, Last known committed idx: 359, Time since last communication: 0.000s
W20250111 09:01:36.031981 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.036262 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.036650 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.039060 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.040249 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.040692 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.040771 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.043421 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.044589 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.046967 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.050370 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.050444 17908 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.051501 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.053582 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.053830 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.056159 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.056224 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.057246 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.058800 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.060123 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.060930 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.061682 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.062237 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.064529 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.067451 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.068148 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.069218 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.070895 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.071303 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.071856 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.076545 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.076754 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.078023 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.079058 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.081467 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.081494 17933 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.081590 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.083057 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.088464 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.089700 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.091101 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.092060 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.092375 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.095147 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.096434 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.098117 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.098426 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.104297 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.104473 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.104612 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.107460 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.110591 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.112304 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.112313 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.114709 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.117066 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.117089 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.121245 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.126199 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.126049 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.126266 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.127808 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.132561 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.132812 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:36.136647 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:36.137084 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:36.137228 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 69 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:36.137595 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 69 FOLLOWER]: Advancing to term 70
W20250111 09:01:36.138566 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:36.138950 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 70 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:36.139732 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 70 candidate_status { last_received { term: 68 index: 324 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:36.139802 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 70 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:36.139868 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 69 FOLLOWER]: Advancing to term 70
I20250111 09:01:36.140220 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 70 candidate_status { last_received { term: 68 index: 324 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:36.140352 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 69 FOLLOWER]: Advancing to term 70
W20250111 09:01:36.140486 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:36.140925 17339 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 70 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 70 because replica has last-logged OpId of term: 68 index: 326, which is greater than that of the candidate, which has last-logged OpId of term: 68 index: 324.
I20250111 09:01:36.141453 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:36.141592 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 72 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:36.141687 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 72 FOLLOWER]: Advancing to term 73
W20250111 09:01:36.142088 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:36.142323 17467 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 70 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 70.
I20250111 09:01:36.142692 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 70 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:36.143184 17961 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 70 FOLLOWER]: Leader election won for term 70
I20250111 09:01:36.143285 17961 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 70 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
W20250111 09:01:36.143393 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Service unavailable: leader is not yet ready
I20250111 09:01:36.143426 17961 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 323, Committed index: 323, Last appended: 68.324, Last appended by leader: 324, Current term: 70, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:36.143815 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Service unavailable: leader is not yet ready
I20250111 09:01:36.144976 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 68 to 70, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 70 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:36.148069 17467 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 70 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 68 index: 323. Preceding OpId from leader: term: 70 index: 326. (index mismatch)
I20250111 09:01:36.148168 17339 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 70 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 68 index: 326. Preceding OpId from leader: term: 70 index: 326. (term mismatch)
I20250111 09:01:36.148219 17339 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 325
I20250111 09:01:36.148247 17339 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted WRITE_OP operation due to leader change: 68.326
W20250111 09:01:36.148344 17339 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Aborted: Op aborted by new leader
I20250111 09:01:36.148530 17961 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 325, Last known committed idx: 322, Time since last communication: 0.000s
I20250111 09:01:36.148839 17956 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 325, Last known committed idx: 324, Time since last communication: 0.000s
I20250111 09:01:36.148936 17956 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer fb270e42d7c9400f8c05bb9565db1972 log is divergent from this leader: its last log entry 68.325 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 324
I20250111 09:01:36.149639 17339 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 324
I20250111 09:01:36.149689 17339 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted WRITE_OP operation due to leader change: 68.325
W20250111 09:01:36.149786 17339 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Aborted: Op aborted by new leader
I20250111 09:01:36.222759 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 73 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:36.222949 17613 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 73 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 72. Current term is 73. Ops: [72.365-72.365]
I20250111 09:01:36.223057 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 73 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:36.223403 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 73 candidate_status { last_received { term: 72 index: 364 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:36.223493 17477 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 72 LEADER]: Stepping down as leader of term 72
I20250111 09:01:36.223520 17968 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: INVALID_TERM, Last received: 72.364, Next index: 365, Last known committed idx: 364, Time since last communication: 0.000s
I20250111 09:01:36.223524 17477 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 72 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:36.223912 17477 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 372, Committed index: 372, Last appended: 72.373, Last appended by leader: 373, Current term: 72, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:36.224020 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 72 FOLLOWER]: Advancing to term 73
I20250111 09:01:36.224996 17477 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 73 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 73 because replica has last-logged OpId of term: 72 index: 373, which is greater than that of the candidate, which has last-logged OpId of term: 72 index: 364.
W20250111 09:01:36.225157 17968 raft_consensus.cc:931] Couldn't advance consensus term.: Illegal state: Can't advance term to: 73 current term: 73 is higher.
W20250111 09:01:36.225457 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.225935 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.229027 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:36.230191 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 73 candidate_status { last_received { term: 72 index: 364 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:36.231484 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 72 FOLLOWER]: Advancing to term 73
W20250111 09:01:36.235375 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.238190 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.238243 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.236017 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.240414 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.244828 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.244923 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.248389 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.251489 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.252270 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.253572 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.255254 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.255504 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.258913 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.259147 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.262018 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.262583 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.262719 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.263927 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.263962 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.268872 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.269074 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.269469 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.272465 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.273495 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.273672 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.275040 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.275285 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.275990 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.276054 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.276844 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.279310 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.279383 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.280299 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.280900 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.281317 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.282748 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.283876 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.284948 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.287242 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.288518 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.288775 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.288882 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.290268 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.290369 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.291640 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.293504 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.295943 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.296507 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.297660 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.298048 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.298048 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.298209 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.298454 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.301242 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.302078 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.303639 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.307832 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.308390 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.308852 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.309644 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.310177 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.311467 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.313800 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.313911 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.315225 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:36.318161 17339 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 73 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 73 because replica has last-logged OpId of term: 72 index: 373, which is greater than that of the candidate, which has last-logged OpId of term: 72 index: 364.
W20250111 09:01:36.318413 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:36.318878 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 73 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:36.319074 17961 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 73 FOLLOWER]: Leader election lost for term 73. Reason: could not achieve majority
W20250111 09:01:36.321848 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.322265 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.323469 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.324534 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.325198 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.326221 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.328287 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.328970 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.329376 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.331676 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.332834 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.336572 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.337651 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.337906 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.338294 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.340961 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.342470 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.343005 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.345290 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.347113 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.349334 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.349422 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.352667 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.353802 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.357177 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.357486 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.357987 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.359304 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.361191 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.363098 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.365550 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.365569 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.368908 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.370445 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.371733 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.371984 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.374799 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.377558 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.379480 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.380909 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.382279 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.383407 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.386801 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.388530 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.389986 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.390692 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.394362 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.396456 17972 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 68 similar messages]
W20250111 09:01:36.396528 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.398313 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.400451 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.402539 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.403726 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.406232 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.406280 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.409368 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.410040 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.414144 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.416409 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.417601 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.420120 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.422796 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.425971 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.426575 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.428673 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.429356 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.433351 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.435616 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.435685 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.439638 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.439759 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.442070 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.446451 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.447623 17970 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 98 similar messages]
W20250111 09:01:36.447672 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.448997 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.454766 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.456372 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.457031 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.458602 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.461624 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.461876 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.466971 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.469877 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.470834 17925 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 87 similar messages]
W20250111 09:01:36.470881 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.474712 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.475862 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.477841 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.480370 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.480437 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.484412 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.487346 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.491612 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.493312 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.497356 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.497427 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.498310 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.503768 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.503813 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.508283 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.511018 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.514496 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.517230 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.517982 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.523658 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.524161 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.527776 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.528713 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.529263 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.536690 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.536990 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.540529 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.541832 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.544858 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.548887 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.550140 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.553326 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.555547 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.563426 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.563989 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.566046 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.567211 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.569290 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.572145 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.573354 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.575841 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.582206 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.589099 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.591032 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.591257 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.591681 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.597177 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.597942 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.600078 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.602260 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.609318 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.618094 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.619192 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.619300 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.619783 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.623690 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.626240 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.626750 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.627949 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.636170 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.643455 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.644637 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.648367 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.649094 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.653272 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.653800 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.654549 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.655042 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.665424 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.671803 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.673910 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.675808 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.679210 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.680822 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.681252 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.683511 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.683692 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.695680 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.700497 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.705375 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.705466 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.711750 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.713085 17970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.713220 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.713640 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.714601 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.725937 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.728515 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.733095 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.736353 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.740924 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.740960 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:36.742584 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:36.743112 17339 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:36.743234 17339 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 70 FOLLOWER]: Starting forced leader election (received explicit request)
W20250111 09:01:36.743252 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:36.743285 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 70 FOLLOWER]: Advancing to term 71
W20250111 09:01:36.743969 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.746563 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.759167 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.761135 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.765142 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.769088 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.771384 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.773535 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.774015 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.774391 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.777125 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.791425 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.792166 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.799604 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.802636 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.802976 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.804423 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.806911 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.808956 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.810181 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.823540 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:36.827193 17339 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 71 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:36.827654 17339 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 71 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
W20250111 09:01:36.827687 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:36.827854 17338 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 71 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 70. Current term is 71. Ops: []
I20250111 09:01:36.827992 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 71 candidate_status { last_received { term: 70 index: 361 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:36.828094 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 70 FOLLOWER]: Advancing to term 71
I20250111 09:01:36.828446 17961 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: INVALID_TERM, Last received: 70.361, Next index: 362, Last known committed idx: 361, Time since last communication: 0.000s
I20250111 09:01:36.828495 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 71 candidate_status { last_received { term: 70 index: 361 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:36.828694 17613 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 70 LEADER]: Stepping down as leader of term 70
I20250111 09:01:36.828738 17613 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 70 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:36.828893 17613 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 361, Committed index: 361, Last appended: 70.361, Last appended by leader: 361, Current term: 70, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:36.829100 17477 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 71 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 71.
I20250111 09:01:36.829190 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 70 FOLLOWER]: Advancing to term 71
I20250111 09:01:36.829376 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 71 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:36.829597 17977 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 71 FOLLOWER]: Leader election won for term 71
I20250111 09:01:36.829706 17977 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 71 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:36.829814 17977 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 361, Committed index: 361, Last appended: 70.361, Last appended by leader: 361, Current term: 71, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:36.830304 17613 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 71 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 71.
W20250111 09:01:36.831446 17961 raft_consensus.cc:931] Couldn't advance consensus term.: Illegal state: Can't advance term to: 71 current term: 71 is higher.
I20250111 09:01:36.831588 17178 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 70 to 71, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 71 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:36.832118 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:36.832237 17338 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 73 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:36.832299 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 73 FOLLOWER]: Advancing to term 74
W20250111 09:01:36.834980 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.835808 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.836804 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.838413 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.839509 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.843251 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.844247 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.860632 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.862417 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.869468 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.870358 17972 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.870429 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.871731 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.873790 17969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.879560 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.880213 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.895862 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.898348 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.904893 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.906206 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.907238 17927 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.907426 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.909968 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.915201 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:36.917783 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:36.921985 17338 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 74 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:36.922324 17338 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 74 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:36.922636 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 74 candidate_status { last_received { term: 72 index: 373 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:36.922760 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 73 FOLLOWER]: Advancing to term 74
I20250111 09:01:36.922744 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 74 candidate_status { last_received { term: 72 index: 373 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:36.922839 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 73 FOLLOWER]: Advancing to term 74
I20250111 09:01:36.923720 17613 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 74 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 74.
I20250111 09:01:36.923720 17477 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 74 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 74.
I20250111 09:01:36.923969 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 74 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:36.924242 17978 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 74 FOLLOWER]: Leader election won for term 74
I20250111 09:01:36.924314 17978 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 74 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:36.924453 17978 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 372, Committed index: 372, Last appended: 72.373, Last appended by leader: 373, Current term: 74, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:36.925773 17178 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 72 to 74, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 74 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:36.932134 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:36.937341 17477 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 74 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 72 index: 373. Preceding OpId from leader: term: 74 index: 375. (index mismatch)
I20250111 09:01:36.937341 17613 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 74 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 72 index: 364. Preceding OpId from leader: term: 74 index: 375. (index mismatch)
I20250111 09:01:36.937693 17977 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 374, Last known committed idx: 364, Time since last communication: 0.000s
I20250111 09:01:36.937880 17978 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 374, Last known committed idx: 372, Time since last communication: 0.000s
W20250111 09:01:36.942051 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.944279 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.946666 17930 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.947038 17973 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:36.950767 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:36.959794 17613 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 71 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 70 index: 361. Preceding OpId from leader: term: 71 index: 363. (index mismatch)
I20250111 09:01:36.960211 17477 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 71 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 70 index: 361. Preceding OpId from leader: term: 71 index: 363. (index mismatch)
I20250111 09:01:36.960677 17977 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 362, Last known committed idx: 361, Time since last communication: 0.000s
I20250111 09:01:36.961130 17977 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 362, Last known committed idx: 361, Time since last communication: 0.000s
W20250111 09:01:36.967857 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:36.971410 17984 mvcc.cc:204] Tried to move back new op lower bound from 7113056653136560128 to 7113056652615892992. Current Snapshot: MvccSnapshot[applied={T|T < 7113056653136560128}]
W20250111 09:01:37.487896 17998 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 65 similar messages]
I20250111 09:01:37.741112 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:37.741837 17475 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:37.742434 17475 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 71 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:37.742530 17475 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 71 FOLLOWER]: Advancing to term 72
I20250111 09:01:37.834604 17475 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 72 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:37.835006 17475 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 72 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:37.835214 17467 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 72 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 71. Current term is 72. Ops: [71.577-71.578]
I20250111 09:01:37.836457 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:37.836417 18004 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: INVALID_TERM, Last received: 71.576, Next index: 577, Last known committed idx: 575, Time since last communication: 0.000s
I20250111 09:01:37.836552 17467 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 74 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:37.836599 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 74 FOLLOWER]: Advancing to term 75
I20250111 09:01:37.836735 18003 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 71 LEADER]: Stepping down as leader of term 71
I20250111 09:01:37.836786 18003 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 71 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:37.836969 18003 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 598, Committed index: 598, Last appended: 71.599, Last appended by leader: 599, Current term: 71, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:37.837257 18003 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 71 FOLLOWER]: Advancing to term 72
W20250111 09:01:37.837622 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:37.838060 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 72 candidate_status { last_received { term: 71 index: 576 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:37.838215 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 71 FOLLOWER]: Advancing to term 72
I20250111 09:01:37.838805 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 72 candidate_status { last_received { term: 71 index: 576 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
W20250111 09:01:37.841594 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.838919 17998 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.006s ago.
W20250111 09:01:37.840442 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.844569 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.845050 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.848246 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.849437 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.851722 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.852766 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.857700 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.858148 18015 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 38 similar messages]
W20250111 09:01:37.858204 18015 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.860818 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.860725 18015 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.861937 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.862291 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.862646 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.866407 18018 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 49 similar messages]
W20250111 09:01:37.866494 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.868398 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.869990 18015 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.869526 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.872175 18019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.872440 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.871244 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.873517 18019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.874711 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.877162 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.877902 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.877929 18019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.879628 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.881819 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.883495 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.883702 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.884334 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.885289 17975 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.885824 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.886408 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.887566 18019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.888947 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.891762 18015 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.893239 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.893430 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.893826 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.895334 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.895680 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.897028 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.897739 18015 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.899683 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.899992 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.900367 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.901364 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.901558 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.903522 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.903964 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.905287 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.905537 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.907848 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.908263 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.908556 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.910039 18015 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.910578 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.910859 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.912020 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.912093 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.912175 18015 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.915910 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.916231 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.917620 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.918146 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.918859 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.919051 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.920301 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.920840 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.921600 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.921977 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.923075 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.925470 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.928102 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.928215 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.928812 18015 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.928969 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.930500 18015 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.930929 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.930974 18016 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.933127 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.935065 18016 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.935838 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.935859 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:37.936555 17467 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 75 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:37.936755 17338 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 72 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 72 because replica has last-logged OpId of term: 71 index: 599, which is greater than that of the candidate, which has last-logged OpId of term: 71 index: 576.
I20250111 09:01:37.936923 17467 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 75 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:37.937309 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 75 candidate_status { last_received { term: 74 index: 622 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:37.937400 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 74 FOLLOWER]: Advancing to term 75
W20250111 09:01:37.937448 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:37.937661 17614 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 72 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 72 because replica has last-logged OpId of term: 71 index: 598, which is greater than that of the candidate, which has last-logged OpId of term: 71 index: 576.
I20250111 09:01:37.937891 17474 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 75 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 74. Current term is 75. Ops: [74.623-74.624]
I20250111 09:01:37.938354 18003 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: INVALID_TERM, Last received: 74.622, Next index: 623, Last known committed idx: 622, Time since last communication: 0.000s
I20250111 09:01:37.938499 17613 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 75 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 75 because replica has last-logged OpId of term: 74 index: 632, which is greater than that of the candidate, which has last-logged OpId of term: 74 index: 622.
I20250111 09:01:37.938530 18003 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 74 LEADER]: Stepping down as leader of term 74
I20250111 09:01:37.938563 18003 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 74 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:37.938680 18003 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 632, Committed index: 632, Last appended: 74.632, Last appended by leader: 632, Current term: 74, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:37.938823 18003 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 74 FOLLOWER]: Advancing to term 75
I20250111 09:01:37.940207 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 72 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:37.940339 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 75 candidate_status { last_received { term: 74 index: 622 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:37.940459 17338 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 75 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 75 because replica has last-logged OpId of term: 74 index: 632, which is greater than that of the candidate, which has last-logged OpId of term: 74 index: 622.
I20250111 09:01:37.940557 17611 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 72 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 71. Current term is 72. Ops: [71.599-71.599]
I20250111 09:01:37.940732 18021 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 72 FOLLOWER]: Leader election lost for term 72. Reason: could not achieve majority
I20250111 09:01:37.940989 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 75 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:37.941206 18021 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 75 FOLLOWER]: Leader election lost for term 75. Reason: could not achieve majority
W20250111 09:01:37.941833 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.942149 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.942490 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.942665 18019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.943310 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.945194 18019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.945313 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.946646 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.947800 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.948115 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.950423 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.952827 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.953157 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.956852 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.957347 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.958400 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.959051 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.959311 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.959810 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.960994 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.962008 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.964463 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.966118 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.967731 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.968506 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.972496 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.972954 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.973049 18016 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.974157 18016 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.976182 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.976246 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.977967 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.979797 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.980304 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.980955 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.984319 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.986867 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.988238 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.989769 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.989845 18019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.990192 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.993320 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.993764 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.994360 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:37.995400 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:37.996141 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.000818 18016 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.002631 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.004424 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.004829 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.005452 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.006376 18016 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.008965 18016 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.008996 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.011464 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.013988 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.014194 18016 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.016361 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.018172 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.021049 18016 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.022176 18016 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.023455 18016 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.023723 18019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.024349 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.028827 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.029417 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.032030 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.032737 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.033315 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.033784 18019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.038285 18019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.041131 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.041471 18019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.042196 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.043329 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.049032 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.049103 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.051903 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.053412 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.053964 18016 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.054200 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.056496 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.059979 18016 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.060333 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.061590 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.062208 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.068876 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.071638 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.072108 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.075816 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.075985 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.077564 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.079150 18019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.079169 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.079243 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.080762 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.082093 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.093174 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.093818 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.094628 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.095089 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.095667 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.098559 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.099836 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.100576 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.101094 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.101397 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.105180 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.113629 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.114238 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.115325 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.116133 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.119946 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.120061 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.121310 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.122923 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.123582 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.125252 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.126403 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.134326 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.136353 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.137068 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.138796 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.140626 18018 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.141031 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.141108 18019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.147217 18019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.148051 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.149288 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.149896 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.156672 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.161288 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.161890 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.162282 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.162551 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.166217 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.166462 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.172372 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.172453 18016 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.173435 18015 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.175335 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.179774 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.183117 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.185135 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.186566 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.189182 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.189671 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.191447 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.199401 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.200829 18010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.200906 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.202436 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.204867 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.207206 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.207563 18015 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.212569 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.214038 18015 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.216949 18015 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.217288 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.226239 18015 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.226338 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.229378 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.231127 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.232632 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.233304 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.233616 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.242022 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.242019 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.244233 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.245646 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.251358 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.252533 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.255148 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.257162 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.258751 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.260768 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.261478 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.268123 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.270275 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.271416 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.273736 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.280112 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.282089 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.283401 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.285295 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.286062 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.288056 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.290616 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.293758 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.299489 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.299547 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.302624 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.308840 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.311141 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.311275 18015 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.314158 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.314975 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.316143 18015 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.320463 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.321518 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.330137 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.332140 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.335179 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.338802 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.340150 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.340224 18019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.344141 18019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.345506 18019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.347962 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.349179 18019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.354142 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.360920 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.361047 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.366842 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:38.369325 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:38.369767 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:38.369875 17611 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 72 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:38.369925 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 72 FOLLOWER]: Advancing to term 73
W20250111 09:01:38.370781 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.371876 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.372802 18015 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.373756 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.377985 18015 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.378687 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.380705 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.383075 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.392748 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.394430 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:38.398330 18015 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.403825 18015 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.403895 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.405246 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.407825 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.409528 18017 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.410015 17925 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.412820 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.417351 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.426795 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.427963 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:38.429245 17611 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 73 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:38.429571 17611 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 73 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:38.429831 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 73 candidate_status { last_received { term: 71 index: 598 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:38.429910 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 73 candidate_status { last_received { term: 71 index: 598 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:38.429946 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 72 FOLLOWER]: Advancing to term 73
I20250111 09:01:38.429999 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 72 FOLLOWER]: Advancing to term 73
I20250111 09:01:38.430817 17474 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 73 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 73.
I20250111 09:01:38.430862 17338 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 73 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 73 because replica has last-logged OpId of term: 71 index: 599, which is greater than that of the candidate, which has last-logged OpId of term: 71 index: 598.
I20250111 09:01:38.431021 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:38.431110 17611 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 75 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:38.431164 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 75 FOLLOWER]: Advancing to term 76
I20250111 09:01:38.431134 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 73 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:38.431294 17961 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 73 FOLLOWER]: Leader election won for term 73
I20250111 09:01:38.431414 17961 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 73 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:38.431550 17961 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 596, Committed index: 596, Last appended: 71.598, Last appended by leader: 598, Current term: 73, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:38.431954 18019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Service unavailable: leader is not yet ready
I20250111 09:01:38.432096 17611 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 76 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:38.432376 17611 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 76 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:38.432632 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 76 candidate_status { last_received { term: 74 index: 632 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:38.432637 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 76 candidate_status { last_received { term: 74 index: 632 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:38.432718 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 75 FOLLOWER]: Advancing to term 76
I20250111 09:01:38.432723 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 75 FOLLOWER]: Advancing to term 76
I20250111 09:01:38.433450 17178 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 71 to 73, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 73 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
W20250111 09:01:38.448762 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:38.524899 17474 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 76 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 76.
I20250111 09:01:38.524899 17338 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 76 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 76.
I20250111 09:01:38.525267 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 76 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:38.525578 18022 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 76 FOLLOWER]: Leader election won for term 76
I20250111 09:01:38.525660 18022 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 76 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:38.525797 18022 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 632, Committed index: 632, Last appended: 74.632, Last appended by leader: 632, Current term: 76, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:38.529995 17178 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 74 to 76, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 76 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:38.889606 17338 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 73 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 71 index: 599. Preceding OpId from leader: term: 73 index: 599. (term mismatch)
I20250111 09:01:38.889678 17338 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 598
I20250111 09:01:38.889701 17338 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted WRITE_OP operation due to leader change: 71.599
W20250111 09:01:38.889804 17338 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Aborted: Op aborted by new leader
I20250111 09:01:38.890334 17338 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39814 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=752, attempt_no=0}) took 1076 ms. Trace:
I20250111 09:01:38.890373 17338 rpcz_store.cc:276] 0111 09:01:37.813758 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:37.813789 (+ 31us) service_pool.cc:225] Handling call
0111 09:01:38.890326 (+1076537us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:37.820069 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:37.820094 (+ 25us) participant_op.cc:199] PREPARE: finished
0111 09:01:37.820110 (+ 16us) participant_op.cc:212] START. Timestamp: 1736586097820087
0111 09:01:37.820112 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:01:37.820266 (+ 154us) log.cc:844] Serialized 66 byte log entry
0111 09:01:37.831839 (+ 11573us) op_driver.cc:464] REPLICATION: finished
0111 09:01:37.831916 (+ 77us) participant_op.cc:265] APPLY: starting
0111 09:01:37.832251 (+ 335us) participant_op.cc:270] APPLY: finished
0111 09:01:37.832267 (+ 16us) log.cc:844] Serialized 15 byte log entry
0111 09:01:37.832368 (+ 101us) participant_op.cc:94] Released txn lock
0111 09:01:37.832373 (+ 5us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:01:37.832546 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 2a4e17d0e2894eaa95b650f1af0ce650
0111 09:01:37.832609 (+ 63us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:37.832610 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:01:37.832611 (+ 1us) tablet.cc:659] Decoding operations
0111 09:01:37.832643 (+ 32us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:01:37.832705 (+ 62us) tablet.cc:698] Row locks acquired
0111 09:01:37.832706 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:01:37.832728 (+ 22us) write_op.cc:270] Start()
0111 09:01:37.832744 (+ 16us) write_op.cc:276] Timestamp: P: 1736586097832706 usec, L: 0
0111 09:01:37.832746 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:01:37.832825 (+ 79us) log.cc:844] Serialized 305 byte log entry
0111 09:01:38.889777 (+1056952us) op_driver.cc:464] REPLICATION: finished
0111 09:01:38.889782 (+ 5us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0111 09:01:38.889785 (+ 3us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:38.889796 (+ 11us) write_op.cc:454] Released schema lock
0111 09:01:38.889801 (+ 5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":32,"txn-participant-registration.run_cpu_time_us":320,"txn-participant-registration.run_wall_time_us":5861,"child_traces":[["op",{"apply.queue_time_us":38,"apply.run_cpu_time_us":439,"apply.run_wall_time_us":610,"prepare.queue_time_us":253,"prepare.run_cpu_time_us":239,"prepare.run_wall_time_us":239,"replication_time_us":11691,"spinlock_wait_cycles":163072,"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":35,"prepare.run_cpu_time_us":323,"prepare.run_wall_time_us":323,"replication_time_us":1056981}]]}]]}
I20250111 09:01:38.891037 18022 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 599, Last known committed idx: 598, Time since last communication: 0.000s
I20250111 09:01:38.893450 17474 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 73 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 71 index: 576. Preceding OpId from leader: term: 73 index: 599. (index mismatch)
I20250111 09:01:38.893777 17961 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 599, Last known committed idx: 575, Time since last communication: 0.000s
W20250111 09:01:38.898412 18032 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 91 similar messages]
W20250111 09:01:38.918248 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.918295 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.918609 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.921777 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.922096 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.924762 18027 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 63 similar messages]
W20250111 09:01:38.925726 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.927954 18062 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 67 similar messages]
W20250111 09:01:38.928042 18062 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:38.929777 17475 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 76 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 74 index: 622. Preceding OpId from leader: term: 76 index: 634. (index mismatch)
I20250111 09:01:38.930218 18039 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 633, Last known committed idx: 622, Time since last communication: 0.000s
W20250111 09:01:38.931195 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:38.931404 17338 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 76 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 74 index: 632. Preceding OpId from leader: term: 76 index: 634. (index mismatch)
W20250111 09:01:38.932386 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.932693 18062 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:38.931420 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:38.934574 17961 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 633, Last known committed idx: 632, Time since last communication: 0.000s
I20250111 09:01:39.000288 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:39.001426 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:39.001543 17338 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 73 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:39.001601 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 73 FOLLOWER]: Advancing to term 74
I20250111 09:01:39.027618 17338 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 74 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:39.027987 17338 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 74 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:39.028148 17340 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 74 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 73. Current term is 74. Ops: [73.628-73.628]
I20250111 09:01:39.028259 17475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 74 candidate_status { last_received { term: 73 index: 627 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:39.028349 17475 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 73 FOLLOWER]: Advancing to term 74
I20250111 09:01:39.029366 17475 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 74 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 74 because replica has last-logged OpId of term: 73 index: 631, which is greater than that of the candidate, which has last-logged OpId of term: 73 index: 627.
I20250111 09:01:39.030047 17467 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 74 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 73. Current term is 74. Ops: [73.632-73.632]
I20250111 09:01:39.030372 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 74 candidate_status { last_received { term: 73 index: 627 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:39.030467 17611 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 73 LEADER]: Stepping down as leader of term 73
I20250111 09:01:39.030500 17611 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 73 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:39.030560 18040 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: INVALID_TERM, Last received: 73.627, Next index: 628, Last known committed idx: 627, Time since last communication: 0.000s
I20250111 09:01:39.030758 17611 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 631, Committed index: 631, Last appended: 73.635, Last appended by leader: 635, Current term: 73, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:39.030908 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 73 FOLLOWER]: Advancing to term 74
I20250111 09:01:39.032662 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:39.032752 17340 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 76 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:39.032797 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 76 FOLLOWER]: Advancing to term 77
W20250111 09:01:39.036207 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.037528 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.037933 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.041083 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.043229 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.048704 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.050805 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.052490 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.056517 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.061414 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.062202 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.064909 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.070190 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.070207 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.070873 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.071843 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.076686 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.076895 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.078125 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.081734 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.082199 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.084893 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.085369 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.088270 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.088310 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.088963 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.089071 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.089843 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.091091 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.092654 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.095204 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.095373 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.095937 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.097640 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.098181 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.098493 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.101034 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.101044 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.102397 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.103294 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.107088 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.107118 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.107477 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.110080 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.110114 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.110574 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.113174 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.113653 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.114475 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.115000 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.116990 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.120249 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.121528 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.121594 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:39.124539 17611 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 74 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 74 because replica has last-logged OpId of term: 73 index: 635, which is greater than that of the candidate, which has last-logged OpId of term: 73 index: 627.
I20250111 09:01:39.124805 17340 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 77 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:39.125088 17337 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 77 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 76. Current term is 77. Ops: [76.674-76.674]
I20250111 09:01:39.125478 17340 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 77 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:39.125854 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 74 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:39.125857 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 77 candidate_status { last_received { term: 76 index: 673 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:39.125974 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 76 FOLLOWER]: Advancing to term 77
I20250111 09:01:39.126226 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 77 candidate_status { last_received { term: 76 index: 673 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:39.126314 17611 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 76 LEADER]: Stepping down as leader of term 76
I20250111 09:01:39.126350 17611 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 76 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:39.126459 17611 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 690, Committed index: 690, Last appended: 76.690, Last appended by leader: 690, Current term: 76, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:39.126569 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 76 FOLLOWER]: Advancing to term 77
I20250111 09:01:39.126952 17477 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 77 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 77 because replica has last-logged OpId of term: 76 index: 690, which is greater than that of the candidate, which has last-logged OpId of term: 76 index: 673.
I20250111 09:01:39.127425 17611 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 77 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 77 because replica has last-logged OpId of term: 76 index: 690, which is greater than that of the candidate, which has last-logged OpId of term: 76 index: 673.
I20250111 09:01:39.127743 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 77 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:39.128055 18076 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 74 FOLLOWER]: Leader election lost for term 74. Reason: could not achieve majority
I20250111 09:01:39.128149 18076 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 77 FOLLOWER]: Leader election lost for term 77. Reason: could not achieve majority
W20250111 09:01:39.128605 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.130115 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.131009 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.131332 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.131740 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.131956 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.133134 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.136739 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.138588 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.142488 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.143538 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.144048 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.144979 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.145359 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.146265 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.148236 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.151907 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.152900 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.155122 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.156837 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.156916 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.157414 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.159847 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.161608 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.164919 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.167024 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.169122 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.170466 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.170954 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.172824 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.174161 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.175550 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.177381 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.180843 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.183451 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.184351 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.187319 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.188232 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.190495 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.191982 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.193178 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.196267 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.199043 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.199059 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.203030 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.206290 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.206682 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.208130 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.212437 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.212782 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.215233 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.217901 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.219973 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.219981 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.223049 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.225777 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.230115 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.231541 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.234700 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.237154 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.238687 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.239629 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.241686 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.242035 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.245687 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.251315 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.253389 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.253757 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.256822 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.258653 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.261183 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.261823 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.263046 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.270320 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.274410 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.274668 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.277417 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.278084 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.281349 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.282737 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.287539 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.287984 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.295360 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.296242 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.298933 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.300201 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.303788 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.305449 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.309564 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.311301 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.315644 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.319362 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.321138 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.323807 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.324960 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.327804 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.329902 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.334232 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.337141 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.338496 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.345609 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.346225 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.348809 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.350088 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.352355 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.357811 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.362583 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.362612 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.367313 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.372479 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.373070 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.374558 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.377722 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.382269 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.385515 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.386932 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.391940 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.399080 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.399256 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.403286 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.404096 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.408828 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.409226 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.414753 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.415771 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.423147 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.428252 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.432101 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.432142 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.436901 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.438339 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.439050 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.442606 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.451864 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.456765 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.460529 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.461254 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.463404 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.464628 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.468082 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.468776 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.476866 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.486902 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.489214 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.490964 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.491456 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.491722 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.497166 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.498059 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.507949 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.516559 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.517098 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.519337 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.521400 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.522096 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.527047 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.531122 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.537969 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.547633 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.548970 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.549296 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.549973 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.553601 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.557181 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.563135 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.568317 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.581207 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.581949 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.582304 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.582818 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.583592 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.588832 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.596127 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.601164 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.612608 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.614362 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.614698 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.615036 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.617223 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.619865 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.631081 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.631086 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.646428 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.646517 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.648180 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.649271 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.649894 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.654211 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.662451 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.667340 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.680437 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.680614 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.681473 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.684255 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.685200 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.687227 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.697490 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.701282 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.715183 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.719223 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.719300 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.719501 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.723498 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.723686 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.734513 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.740581 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.749265 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.753415 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.753496 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.756361 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.758183 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.759302 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.771421 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.776539 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.788391 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.789113 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.792430 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.796129 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.798189 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.798308 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.807935 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.814527 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.827760 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.828621 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.829814 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.836736 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.837818 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.840636 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.844388 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.852088 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.866014 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.867934 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.869124 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.874841 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.876255 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.881961 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.885149 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.893093 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.906281 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.907936 18063 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 80 similar messages]
W20250111 09:01:39.907982 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.908388 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.916571 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.919320 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.923889 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.926949 18020 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 71 similar messages]
W20250111 09:01:39.926998 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.933993 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.947316 18020 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.947474 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.948441 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:39.955161 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:39.955653 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:39.955821 17477 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 74 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:39.955893 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 74 FOLLOWER]: Advancing to term 75
W20250111 09:01:39.960412 18064 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 75 similar messages]
W20250111 09:01:39.960538 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.960757 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.967814 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.969183 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.977386 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:39.987155 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.989668 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:39.991932 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:40.001804 17477 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 75 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.002184 17477 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 75 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:40.002537 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 75 candidate_status { last_received { term: 73 index: 631 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:40.002568 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 75 candidate_status { last_received { term: 73 index: 631 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:40.002669 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 74 FOLLOWER]: Advancing to term 75
I20250111 09:01:40.002681 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 74 FOLLOWER]: Advancing to term 75
I20250111 09:01:40.003392 17340 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 75 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 75.
I20250111 09:01:40.003664 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:40.003695 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 75 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:40.003768 17477 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 77 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:40.003775 17611 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 75 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 75 because replica has last-logged OpId of term: 73 index: 635, which is greater than that of the candidate, which has last-logged OpId of term: 73 index: 631.
I20250111 09:01:40.003834 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 77 FOLLOWER]: Advancing to term 78
I20250111 09:01:40.004241 18077 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 75 FOLLOWER]: Leader election won for term 75
I20250111 09:01:40.004382 18077 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 75 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
W20250111 09:01:40.004513 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:40.004566 18077 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 631, Committed index: 631, Last appended: 73.631, Last appended by leader: 631, Current term: 75, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.004776 17477 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 78 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.005084 17477 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 78 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:40.005502 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 78 candidate_status { last_received { term: 76 index: 690 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:40.005596 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 77 FOLLOWER]: Advancing to term 78
I20250111 09:01:40.005942 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 78 candidate_status { last_received { term: 76 index: 690 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:40.006033 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 77 FOLLOWER]: Advancing to term 78
I20250111 09:01:40.006386 17611 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 78 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 78.
I20250111 09:01:40.006803 17340 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 78 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 78.
I20250111 09:01:40.007061 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 78 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:40.007254 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 73 to 75, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 75 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:40.007475 17611 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 75 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 73 index: 635. Preceding OpId from leader: term: 75 index: 633. (term mismatch)
I20250111 09:01:40.007536 17611 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 632
I20250111 09:01:40.007570 17611 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 73.633
I20250111 09:01:40.007558 18078 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 78 FOLLOWER]: Leader election won for term 78
I20250111 09:01:40.007624 18078 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 78 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:40.007648 17340 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 75 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 73 index: 627. Preceding OpId from leader: term: 75 index: 633. (index mismatch)
W20250111 09:01:40.007678 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:01:40.007752 18078 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 690, Committed index: 690, Last appended: 76.690, Last appended by leader: 690, Current term: 78, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.007881 17611 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 73.634
W20250111 09:01:40.007967 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:01:40.008121 17611 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 73.635
I20250111 09:01:40.008100 18080 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 632, Last known committed idx: 627, Time since last communication: 0.000s
W20250111 09:01:40.008211 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Aborted: Op aborted by new leader
I20250111 09:01:40.008715 18077 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 632, Last known committed idx: 631, Time since last communication: 0.000s
I20250111 09:01:40.008797 18077 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 73.632 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 631
I20250111 09:01:40.009424 17611 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 631
I20250111 09:01:40.009481 17611 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 73.632
W20250111 09:01:40.009567 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Aborted: Op aborted by new leader
W20250111 09:01:40.010756 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.013119 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.013321 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.014384 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:40.018579 17178 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 76 to 78, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 78 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:40.019146 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.023993 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.024178 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.028659 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.031250 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.032225 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:40.036226 17611 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 78 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 76 index: 690. Preceding OpId from leader: term: 78 index: 692. (index mismatch)
I20250111 09:01:40.036512 17340 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 78 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 76 index: 673. Preceding OpId from leader: term: 78 index: 692. (index mismatch)
I20250111 09:01:40.036602 18080 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 691, Last known committed idx: 690, Time since last communication: 0.000s
I20250111 09:01:40.036815 18077 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 691, Last known committed idx: 671, Time since last communication: 0.000s
I20250111 09:01:40.068823 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:40.070230 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:40.074025 17611 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 75 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:40.074083 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 75 FOLLOWER]: Advancing to term 76
I20250111 09:01:40.074957 17611 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 76 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.075412 17611 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 76 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:40.076149 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 76 candidate_status { last_received { term: 75 index: 653 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:40.077512 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 76 candidate_status { last_received { term: 75 index: 653 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:40.077600 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 75 FOLLOWER]: Advancing to term 76
I20250111 09:01:40.078881 17338 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 76 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 76.
I20250111 09:01:40.079144 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 76 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:01:40.079339 18101 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 76 FOLLOWER]: Leader election won for term 76
I20250111 09:01:40.079397 18101 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 76 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:40.079493 18101 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 652, Committed index: 652, Last appended: 75.653, Last appended by leader: 653, Current term: 76, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.079620 17337 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 76 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 75. Current term is 76. Ops: [75.651-75.652]
I20250111 09:01:40.080811 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:40.080965 17611 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 78 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:40.081084 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 78 FOLLOWER]: Advancing to term 79
I20250111 09:01:40.081197 17614 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 76 LEADER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 75. Current term is 76. Ops: [75.654-75.656]
I20250111 09:01:40.081884 17611 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 79 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.081928 18077 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: INVALID_TERM, Last received: 75.653, Next index: 654, Last known committed idx: 652, Time since last communication: 0.000s
I20250111 09:01:40.081872 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 75 to 76, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 76 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:40.082059 18077 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 76.654 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 652
I20250111 09:01:40.082119 18077 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 75 LEADER]: Stepping down as leader of term 75
I20250111 09:01:40.082167 18077 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 75 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:40.082289 18077 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 653, Committed index: 653, Last appended: 75.657, Last appended by leader: 657, Current term: 75, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.082427 18077 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 75 FOLLOWER]: Advancing to term 76
I20250111 09:01:40.082803 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 79 candidate_status { last_received { term: 78 index: 695 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:40.082913 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 78 FOLLOWER]: Advancing to term 79
I20250111 09:01:40.083751 17614 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 79 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 78. Current term is 79. Ops: [78.696-78.696]
I20250111 09:01:40.083828 17337 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 79 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 79.
I20250111 09:01:40.084080 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 79 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:40.084180 18077 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: INVALID_TERM, Last received: 78.695, Next index: 696, Last known committed idx: 695, Time since last communication: 0.000s
I20250111 09:01:40.084280 18039 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 79 FOLLOWER]: Leader election won for term 79
I20250111 09:01:40.084297 18077 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 78 LEADER]: Stepping down as leader of term 78
I20250111 09:01:40.084335 18077 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 78 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:40.084344 18039 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 79 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:40.084424 17611 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 79 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:40.084429 18077 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 695, Committed index: 695, Last appended: 78.696, Last appended by leader: 696, Current term: 78, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.084465 18039 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 695, Committed index: 695, Last appended: 78.695, Last appended by leader: 695, Current term: 79, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.084563 18077 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 78 FOLLOWER]: Advancing to term 79
I20250111 09:01:40.085858 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 79 candidate_status { last_received { term: 78 index: 695 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:40.086056 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 78 to 79, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 79 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
W20250111 09:01:40.086503 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.086694 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:40.088083 17467 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 79 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 79 because replica has last-logged OpId of term: 78 index: 696, which is greater than that of the candidate, which has last-logged OpId of term: 78 index: 695.
W20250111 09:01:40.091948 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.092015 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:40.092645 17337 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 79 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 78. Current term is 79. Ops: [78.696-78.696]
W20250111 09:01:40.093257 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.095914 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:40.096297 17467 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 76 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 75 index: 657. Preceding OpId from leader: term: 76 index: 655. (term mismatch)
I20250111 09:01:40.096359 17467 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 654
I20250111 09:01:40.096395 17467 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 75.655
W20250111 09:01:40.096467 17467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
I20250111 09:01:40.096609 17467 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 75.656
W20250111 09:01:40.096688 17467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
I20250111 09:01:40.096834 17467 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 75.657
W20250111 09:01:40.096941 17467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
I20250111 09:01:40.097472 18039 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 654, Last known committed idx: 653, Time since last communication: 0.000s
I20250111 09:01:40.097566 18039 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer 262623ac6df745c785ce267ea39bd265 log is divergent from this leader: its last log entry 75.654 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 653
I20250111 09:01:40.097939 17340 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 76 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 75 index: 650. Preceding OpId from leader: term: 76 index: 655. (index mismatch)
I20250111 09:01:40.098012 17467 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 653
I20250111 09:01:40.098053 17467 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 75.654
W20250111 09:01:40.098133 17467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
W20250111 09:01:40.098469 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:40.100646 18100 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 654, Last known committed idx: 650, Time since last communication: 0.000s
I20250111 09:01:40.101102 17339 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 79 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 78 index: 695. Preceding OpId from leader: term: 79 index: 697. (index mismatch)
W20250111 09:01:40.101362 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:40.102649 18039 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 696, Last known committed idx: 695, Time since last communication: 0.000s
W20250111 09:01:40.103073 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.103618 18075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.104475 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:40.106037 17477 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 79 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 78 index: 696. Preceding OpId from leader: term: 79 index: 697. (index mismatch)
I20250111 09:01:40.106385 18101 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 696, Last known committed idx: 695, Time since last communication: 0.000s
I20250111 09:01:40.106480 18101 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer 262623ac6df745c785ce267ea39bd265 log is divergent from this leader: its last log entry 78.696 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 695
I20250111 09:01:40.107664 17467 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 695
I20250111 09:01:40.107710 17467 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 78.696
W20250111 09:01:40.107802 17467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
I20250111 09:01:40.112865 18102 mvcc.cc:204] Tried to move back new op lower bound from 7113056666008694784 to 7113056665947025408. Current Snapshot: MvccSnapshot[applied={T|T < 7113056666008694784}]
I20250111 09:01:40.466487 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:40.467331 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:40.467571 17340 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 76 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:40.467633 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 76 FOLLOWER]: Advancing to term 77
I20250111 09:01:40.468775 17340 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 77 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.470093 17340 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 77 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:40.470496 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 77 candidate_status { last_received { term: 76 index: 760 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:40.470598 17611 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 76 LEADER]: Stepping down as leader of term 76
I20250111 09:01:40.470642 17611 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 76 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:40.470753 17611 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 759, Committed index: 759, Last appended: 76.760, Last appended by leader: 760, Current term: 76, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.471478 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 77 candidate_status { last_received { term: 76 index: 760 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:40.471592 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:40.471623 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 76 FOLLOWER]: Advancing to term 77
I20250111 09:01:40.472679 17474 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 77 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 77.
I20250111 09:01:40.472718 17340 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 79 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:40.472786 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 79 FOLLOWER]: Advancing to term 80
I20250111 09:01:40.472990 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 77 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:40.473212 18082 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 77 FOLLOWER]: Leader election won for term 77
I20250111 09:01:40.473399 18082 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 77 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:40.473515 18082 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 759, Committed index: 759, Last appended: 76.760, Last appended by leader: 760, Current term: 77, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.475289 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 76 FOLLOWER]: Advancing to term 77
I20250111 09:01:40.476305 17611 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 77 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 77.
I20250111 09:01:40.476188 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 76 to 77, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 77 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:40.477456 17340 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 80 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.477964 17340 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 80 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:40.478566 17475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 80 candidate_status { last_received { term: 79 index: 795 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:40.478667 17475 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 79 FOLLOWER]: Advancing to term 80
I20250111 09:01:40.479669 17475 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 80 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 80 because replica has last-logged OpId of term: 79 index: 797, which is greater than that of the candidate, which has last-logged OpId of term: 79 index: 795.
I20250111 09:01:40.481436 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 80 candidate_status { last_received { term: 79 index: 795 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
W20250111 09:01:40.481838 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:40.482112 17475 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 80 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 79. Current term is 80. Ops: [79.798-79.800]
I20250111 09:01:40.482316 17611 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 79 LEADER]: Stepping down as leader of term 79
I20250111 09:01:40.482373 17611 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 79 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:40.482477 17611 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 797, Committed index: 797, Last appended: 79.800, Last appended by leader: 800, Current term: 79, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.482604 18116 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: INVALID_TERM, Last received: 79.797, Next index: 798, Last known committed idx: 794, Time since last communication: 0.000s
W20250111 09:01:40.482643 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.482838 18087 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.004s ago.
W20250111 09:01:40.483243 18097 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.004s ago.
I20250111 09:01:40.482841 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 79 FOLLOWER]: Advancing to term 80
I20250111 09:01:40.483893 17338 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 80 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 79. Current term is 80. Ops: [79.796-79.797]
W20250111 09:01:40.484103 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:40.484684 17611 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 80 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 80 because replica has last-logged OpId of term: 79 index: 800, which is greater than that of the candidate, which has last-logged OpId of term: 79 index: 795.
I20250111 09:01:40.485141 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 80 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:40.487599 18121 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 80 FOLLOWER]: Leader election lost for term 80. Reason: could not achieve majority
W20250111 09:01:40.489996 18120 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.490671 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:40.491592 17611 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 77 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 76 index: 760. Preceding OpId from leader: term: 77 index: 762. (index mismatch)
I20250111 09:01:40.491883 18121 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 761, Last known committed idx: 759, Time since last communication: 0.000s
W20250111 09:01:40.492245 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.492092 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.493973 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:40.494874 18103 mvcc.cc:204] Tried to move back new op lower bound from 7113056667607142400 to 7113056667540721664. Current Snapshot: MvccSnapshot[applied={T|T < 7113056667607142400}]
I20250111 09:01:40.496276 17474 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 77 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 76 index: 759. Preceding OpId from leader: term: 77 index: 762. (index mismatch)
W20250111 09:01:40.496510 18120 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:40.498291 18122 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 761, Last known committed idx: 759, Time since last communication: 0.000s
W20250111 09:01:40.498378 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.499361 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.499688 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:40.508787 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
W20250111 09:01:40.508975 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:40.509294 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
W20250111 09:01:40.509382 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:40.509388 17467 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 77 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:40.509441 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 77 FOLLOWER]: Advancing to term 78
I20250111 09:01:40.510265 17467 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 78 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.510591 17467 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 78 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
W20250111 09:01:40.510663 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.510908 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:40.511566 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 78 candidate_status { last_received { term: 77 index: 765 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:40.511899 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 78 candidate_status { last_received { term: 77 index: 765 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:40.511983 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 77 FOLLOWER]: Advancing to term 78
I20250111 09:01:40.512548 17475 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 78 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 77. Current term is 78. Ops: [77.766-77.768]
I20250111 09:01:40.512734 17611 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 78 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 78 because replica has last-logged OpId of term: 77 index: 768, which is greater than that of the candidate, which has last-logged OpId of term: 77 index: 765.
I20250111 09:01:40.512560 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:40.512887 17467 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 80 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:40.512943 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 80 FOLLOWER]: Advancing to term 81
I20250111 09:01:40.513125 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 78 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:40.513356 17611 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 78 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 77. Current term is 78. Ops: [77.769-77.769]
I20250111 09:01:40.513813 17467 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 81 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.514122 17467 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 81 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:40.514477 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 81 candidate_status { last_received { term: 79 index: 797 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:40.514605 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 80 FOLLOWER]: Advancing to term 81
I20250111 09:01:40.515751 17338 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 81 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 81.
I20250111 09:01:40.515980 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 81 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:40.516263 18077 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 81 FOLLOWER]: Leader election won for term 81
I20250111 09:01:40.516337 18077 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 81 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:40.516448 18077 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 794, Committed index: 794, Last appended: 79.797, Last appended by leader: 797, Current term: 81, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:40.517915 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.518304 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:40.518484 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 79 to 81, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 81 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:40.518848 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:40.519472 18121 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: INVALID_TERM, Last received: 77.768, Next index: 769, Last known committed idx: 768, Time since last communication: 0.000s
I20250111 09:01:40.519577 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 81 candidate_status { last_received { term: 79 index: 797 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:40.519598 18121 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 77 LEADER]: Stepping down as leader of term 77
I20250111 09:01:40.519637 18121 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 77 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:40.519663 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 80 FOLLOWER]: Advancing to term 81
I20250111 09:01:40.519754 18121 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 768, Committed index: 768, Last appended: 77.771, Last appended by leader: 771, Current term: 77, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.519884 18121 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 77 FOLLOWER]: Advancing to term 78
I20250111 09:01:40.521327 18104 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 78 FOLLOWER]: Leader election lost for term 78. Reason: could not achieve majority
I20250111 09:01:40.521431 17611 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 81 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 81 because replica has last-logged OpId of term: 79 index: 800, which is greater than that of the candidate, which has last-logged OpId of term: 79 index: 797.
W20250111 09:01:40.522029 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.522343 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.526710 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:40.528178 17340 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 81 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 79 index: 795. Preceding OpId from leader: term: 81 index: 799. (index mismatch)
I20250111 09:01:40.528188 17611 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 81 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 79 index: 800. Preceding OpId from leader: term: 81 index: 799. (term mismatch)
I20250111 09:01:40.528236 17611 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 798
I20250111 09:01:40.528270 17611 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 79.799
W20250111 09:01:40.528339 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:01:40.528465 18080 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 798, Last known committed idx: 794, Time since last communication: 0.000s
I20250111 09:01:40.528811 17611 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 79.800
W20250111 09:01:40.528905 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:01:40.530063 18080 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 798, Last known committed idx: 797, Time since last communication: 0.000s
I20250111 09:01:40.530145 18080 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 79.798 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 797
I20250111 09:01:40.530820 17611 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 797
I20250111 09:01:40.530889 17611 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 79.798
W20250111 09:01:40.530990 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
W20250111 09:01:40.556943 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.561652 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.563892 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.566891 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.570238 18120 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.571990 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.574815 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.577307 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.578730 18120 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.580106 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.580829 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.580960 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.583070 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.586639 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.587963 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.589676 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.590420 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.591591 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.594043 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.595546 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.595945 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.597883 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.599107 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.600340 18120 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.600703 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.603948 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.604027 18120 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.606051 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.607088 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.608547 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.609941 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.610787 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.612128 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.613384 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.616652 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.619179 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.621578 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.622014 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.622676 18120 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.624890 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.626874 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.627470 18120 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.630679 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.630847 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.637521 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.638208 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.639967 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.640005 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.640329 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.642977 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.651060 18120 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.652333 18120 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.652771 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.653231 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.653396 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.659157 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.661630 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.665760 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.666379 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.667694 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.667742 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.671324 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.676753 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.678200 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.681118 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.682296 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.684770 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.686668 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.691457 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.692703 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.695371 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.697283 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.702838 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.703251 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.706533 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.710912 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.712306 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.714838 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.717945 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.721118 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.721354 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.726974 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.729126 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.732964 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.735662 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.736394 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.738831 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.744464 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.746214 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.753294 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.754009 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.754387 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.759059 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.762053 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.768518 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.773204 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.774236 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.776993 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.780145 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.784312 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.788570 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.791410 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.794852 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.796734 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.802336 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.807024 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.808244 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.815636 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.817965 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.820142 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.823704 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.831736 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.832993 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.836663 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.838618 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.845222 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.848274 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.852577 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.854926 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.857843 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.863165 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.870201 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.871002 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.876735 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.878093 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:40.884493 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.886927 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.895624 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.896054 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.904021 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.904376 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.910260 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:40.912348 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:40.912808 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:40.912935 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 78 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:40.913015 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 78 FOLLOWER]: Advancing to term 79
W20250111 09:01:40.914042 18063 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 57 similar messages]
W20250111 09:01:40.914112 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:40.914099 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 79 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.914466 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 79 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:40.914757 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 79 candidate_status { last_received { term: 77 index: 768 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:40.914892 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 78 FOLLOWER]: Advancing to term 79
I20250111 09:01:40.915113 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:40.915212 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 81 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:40.915266 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 81 FOLLOWER]: Advancing to term 82
I20250111 09:01:40.915807 17340 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 79 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 79 because replica has last-logged OpId of term: 77 index: 771, which is greater than that of the candidate, which has last-logged OpId of term: 77 index: 768.
I20250111 09:01:40.915918 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 82 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.916049 17475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 79 candidate_status { last_received { term: 77 index: 768 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:40.916165 17475 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 78 FOLLOWER]: Advancing to term 79
I20250111 09:01:40.916219 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 82 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:40.916478 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 82 candidate_status { last_received { term: 81 index: 818 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:40.916481 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 82 candidate_status { last_received { term: 81 index: 818 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:40.916566 17467 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 81 LEADER]: Stepping down as leader of term 81
I20250111 09:01:40.916565 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 81 FOLLOWER]: Advancing to term 82
I20250111 09:01:40.916612 17467 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 81 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:40.916749 17467 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 818, Committed index: 818, Last appended: 81.818, Last appended by leader: 818, Current term: 81, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.916922 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 81 FOLLOWER]: Advancing to term 82
I20250111 09:01:40.917088 17475 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 79 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 79.
I20250111 09:01:40.917322 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 79 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:40.917340 17340 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 82 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 82.
I20250111 09:01:40.917528 18101 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 79 FOLLOWER]: Leader election won for term 79
I20250111 09:01:40.917622 18101 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 79 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:40.917805 17467 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 82 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 82.
I20250111 09:01:40.917768 18101 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 768, Committed index: 768, Last appended: 77.768, Last appended by leader: 768, Current term: 79, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.918082 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 82 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:40.918238 18039 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 82 FOLLOWER]: Leader election won for term 82
I20250111 09:01:40.918320 18039 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 82 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:40.918416 18039 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 818, Committed index: 818, Last appended: 81.818, Last appended by leader: 818, Current term: 82, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:40.919241 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 77 to 79, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 79 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
W20250111 09:01:40.920737 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:40.922008 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 81 to 82, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 82 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
W20250111 09:01:40.924016 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.929206 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.931205 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:40.936877 17467 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 79 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 77 index: 765. Preceding OpId from leader: term: 79 index: 770. (index mismatch)
I20250111 09:01:40.936897 17340 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 79 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 77 index: 771. Preceding OpId from leader: term: 79 index: 770. (term mismatch)
I20250111 09:01:40.936940 17340 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 769
I20250111 09:01:40.936964 17340 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted WRITE_OP operation due to leader change: 77.770
W20250111 09:01:40.937070 17340 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Aborted: Op aborted by new leader
I20250111 09:01:40.937188 18101 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 769, Last known committed idx: 764, Time since last communication: 0.000s
I20250111 09:01:40.937275 17340 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 77.771
W20250111 09:01:40.937359 17340 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Aborted: Op aborted by new leader
I20250111 09:01:40.937820 18101 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 769, Last known committed idx: 768, Time since last communication: 0.000s
I20250111 09:01:40.937903 18101 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer fb270e42d7c9400f8c05bb9565db1972 log is divergent from this leader: its last log entry 77.769 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 768
I20250111 09:01:40.938405 17340 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 768
I20250111 09:01:40.938463 17340 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted WRITE_OP operation due to leader change: 77.769
W20250111 09:01:40.938556 17340 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Aborted: Op aborted by new leader
W20250111 09:01:40.940755 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.945160 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.947027 18117 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 78 similar messages]
W20250111 09:01:40.949692 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.950356 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:40.956894 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:40.967734 17475 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 82 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 81 index: 818. Preceding OpId from leader: term: 82 index: 820. (index mismatch)
I20250111 09:01:40.969826 18111 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 819, Last known committed idx: 818, Time since last communication: 0.000s
I20250111 09:01:40.972195 17340 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 82 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 81 index: 818. Preceding OpId from leader: term: 82 index: 820. (index mismatch)
W20250111 09:01:40.972510 18079 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 68 similar messages]
I20250111 09:01:40.978673 18039 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 819, Last known committed idx: 818, Time since last communication: 0.000s
I20250111 09:01:41.021032 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:41.021467 17337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:41.021586 17337 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 79 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:41.021651 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 79 FOLLOWER]: Advancing to term 80
I20250111 09:01:41.022517 17337 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 80 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:41.022850 17337 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 80 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:41.023092 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 80 candidate_status { last_received { term: 79 index: 794 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:41.023440 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 79 FOLLOWER]: Advancing to term 80
I20250111 09:01:41.023697 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:41.023782 17340 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 82 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:41.023845 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 82 FOLLOWER]: Advancing to term 83
I20250111 09:01:41.024585 17474 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 80 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 80 because replica has last-logged OpId of term: 79 index: 796, which is greater than that of the candidate, which has last-logged OpId of term: 79 index: 794.
I20250111 09:01:41.024626 17340 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 83 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:41.024825 17338 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 83 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 82. Current term is 83. Ops: [82.832-82.838]
I20250111 09:01:41.024966 17339 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 80 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 79. Current term is 80. Ops: [79.795-79.797]
I20250111 09:01:41.025192 17340 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 83 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:41.026939 17467 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 80 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 79. Current term is 80. Ops: [79.797-79.797]
I20250111 09:01:41.027149 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 80 candidate_status { last_received { term: 79 index: 794 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:41.027266 17614 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 79 LEADER]: Stepping down as leader of term 79
I20250111 09:01:41.027319 17614 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 79 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:41.027478 17614 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 796, Committed index: 796, Last appended: 79.797, Last appended by leader: 797, Current term: 79, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:41.027609 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 79 FOLLOWER]: Advancing to term 80
I20250111 09:01:41.025480 17475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 83 candidate_status { last_received { term: 82 index: 831 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:41.028358 17614 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 80 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 80 because replica has last-logged OpId of term: 79 index: 797, which is greater than that of the candidate, which has last-logged OpId of term: 79 index: 794.
I20250111 09:01:41.028721 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 83 candidate_status { last_received { term: 82 index: 831 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
W20250111 09:01:41.028815 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:41.028813 17614 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 82 LEADER]: Stepping down as leader of term 82
I20250111 09:01:41.028898 17614 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 82 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:41.028993 17614 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 837, Committed index: 837, Last appended: 82.839, Last appended by leader: 839, Current term: 82, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:41.029096 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 82 FOLLOWER]: Advancing to term 83
I20250111 09:01:41.029899 17614 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 83 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 83 because replica has last-logged OpId of term: 82 index: 839, which is greater than that of the candidate, which has last-logged OpId of term: 82 index: 831.
I20250111 09:01:41.030866 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 80 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:41.031085 18082 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 80 FOLLOWER]: Leader election lost for term 80. Reason: could not achieve majority
I20250111 09:01:41.031508 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 83 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:41.031704 18082 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 83 FOLLOWER]: Leader election lost for term 83. Reason: could not achieve majority
W20250111 09:01:41.033900 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.033924 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.034766 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.035118 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.035323 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.035768 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.036381 18125 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.036387 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.037523 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.040232 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.040712 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.041118 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.041162 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.041579 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.041982 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.042670 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.044359 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.046157 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.049283 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.049369 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.050642 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.051363 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.051712 18125 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.053380 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.056118 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.056231 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.057054 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.058450 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.059777 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.061420 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.061498 18065 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.062700 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.062984 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.063870 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.066655 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.067028 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.067152 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.068691 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.069505 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.070164 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.071264 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.071574 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.071992 18125 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.073856 18125 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.076270 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.077224 18125 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.078127 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.079108 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.080185 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.080729 18125 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.080876 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.081794 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.082959 18125 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.084142 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.085564 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.087711 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.087775 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.089804 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.089885 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.091158 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.091884 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.092726 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.096129 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.097600 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.097668 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.097728 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.097987 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.099625 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.103060 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.104287 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.104995 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.107411 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.108450 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.108891 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.109711 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.109880 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.113045 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.113328 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.115917 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.117952 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.120759 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.121879 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.122954 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.123054 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.123739 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.125202 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.127763 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.129603 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.130296 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.132190 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.135499 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.137152 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.137152 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.139334 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.140183 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.141314 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.145532 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.145624 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.147536 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.148810 18125 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.150804 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.153586 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.153944 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.154270 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.157761 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.160591 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.160945 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.162889 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.164609 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.165482 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.169592 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.170017 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.170228 18125 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.172634 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.178503 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.178790 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.180121 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.181581 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.181605 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.186771 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.187652 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.188349 18125 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.190935 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.193663 18125 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.195502 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.196130 18125 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.198098 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.201905 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.203207 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.205299 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.206931 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.209702 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.211935 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.211946 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.213630 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.215245 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.220116 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.220173 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.224980 18125 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.226446 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.228324 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.229210 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.231925 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.232283 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.235497 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.240231 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.240296 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.245486 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.247929 18125 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.248466 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.250065 18125 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.250181 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.250731 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.254611 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.260221 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.264626 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.269447 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.269523 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.270318 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.270901 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.272303 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.272418 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.274912 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.282691 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.289057 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.290087 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.291117 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.291570 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.292682 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.293087 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.297710 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.299055 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.308071 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.312160 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.313076 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.314399 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.315268 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.316692 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.317466 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.320384 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.321429 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.330983 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.335719 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.338351 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.339205 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.339629 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.340858 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.342221 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.344686 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.347496 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.357332 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.359745 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.362542 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.362628 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.366222 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.366683 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.366935 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.370467 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.375134 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.382959 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.387367 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.388077 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.389209 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.391476 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.393471 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.394783 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.396270 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.399384 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.411368 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.412693 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.415009 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.415962 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.417378 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.418448 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.423134 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.423144 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.429302 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.439105 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.441069 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.442168 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.444463 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.445678 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.446624 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.451341 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.453644 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.459095 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.470119 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.471053 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.472178 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.472570 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.474336 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.475994 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.480758 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.483103 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.488481 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.497113 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.502264 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.504025 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.504302 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.505501 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.506775 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.510576 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.514637 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.520093 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.529620 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.534839 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.535920 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.535956 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.537993 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.538087 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.542375 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.547027 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.553776 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.559304 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.564713 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.565850 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.569991 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.571185 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.571254 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.575085 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.580634 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.587374 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.593739 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.596673 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.597983 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.601900 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:41.605432 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
W20250111 09:01:41.605476 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:41.605912 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:41.606050 17477 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 80 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:41.606112 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 80 FOLLOWER]: Advancing to term 81
W20250111 09:01:41.606866 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:41.607035 17477 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 81 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:41.607434 17477 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 81 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:41.608071 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 81 candidate_status { last_received { term: 79 index: 796 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:41.608071 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 81 candidate_status { last_received { term: 79 index: 796 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:41.608170 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 80 FOLLOWER]: Advancing to term 81
I20250111 09:01:41.608170 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 80 FOLLOWER]: Advancing to term 81
W20250111 09:01:41.608961 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:41.608989 17337 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 81 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 81.
I20250111 09:01:41.609059 17614 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 81 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 81 because replica has last-logged OpId of term: 79 index: 797, which is greater than that of the candidate, which has last-logged OpId of term: 79 index: 796.
I20250111 09:01:41.609320 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:41.609405 17477 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 82 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:41.609457 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 82 FOLLOWER]: Advancing to term 83
I20250111 09:01:41.609436 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 81 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters: b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:41.609665 18127 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 81 FOLLOWER]: Leader election won for term 81
I20250111 09:01:41.609833 18127 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 81 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:41.609992 18127 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 796, Committed index: 796, Last appended: 79.796, Last appended by leader: 796, Current term: 81, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:41.610234 17477 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 83 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:41.610486 17477 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 83 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:41.610772 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 83 candidate_status { last_received { term: 82 index: 838 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:41.610862 17614 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 83 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 83 because replica has last-logged OpId of term: 82 index: 839, which is greater than that of the candidate, which has last-logged OpId of term: 82 index: 838.
I20250111 09:01:41.611141 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 83 candidate_status { last_received { term: 82 index: 838 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:41.611253 17337 raft_consensus.cc:2388] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 83 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 in current term 83: Already voted for candidate fb270e42d7c9400f8c05bb9565db1972 in this term.
I20250111 09:01:41.611476 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 83 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:41.611462 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 79 to 81, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 81 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:41.611632 18127 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 83 FOLLOWER]: Leader election lost for term 83. Reason: could not achieve majority
I20250111 09:01:41.616465 17614 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 81 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 79 index: 797. Preceding OpId from leader: term: 81 index: 798. (index mismatch)
I20250111 09:01:41.616477 17337 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 81 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 79 index: 794. Preceding OpId from leader: term: 81 index: 798. (index mismatch)
I20250111 09:01:41.616761 18127 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 797, Last known committed idx: 792, Time since last communication: 0.000s
I20250111 09:01:41.616873 18128 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 797, Last known committed idx: 796, Time since last communication: 0.000s
I20250111 09:01:41.616964 18128 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 79.797 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 796
I20250111 09:01:41.617453 17614 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 796
I20250111 09:01:41.617492 17614 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 79.797
W20250111 09:01:41.617564 17614 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
W20250111 09:01:41.623302 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.628891 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.629099 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.633291 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.634953 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.639338 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.639434 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.642563 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.643028 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.644542 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.646989 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.647383 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.652127 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.653199 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.655078 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.655545 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.659052 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.659233 18141 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.661659 18141 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.663596 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.664836 18141 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.666314 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.668622 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.669327 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.672742 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.674178 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.675168 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.675618 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.675963 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.678855 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.679850 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.680421 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.685809 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.685899 18141 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.687260 18141 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.687574 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.696777 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.698230 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.698298 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.699821 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.700338 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.706957 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.709297 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.709606 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.712481 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.714123 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.714251 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.715157 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.719164 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.721233 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.723716 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.728307 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.730366 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.733474 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:41.735109 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.741195 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.742283 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.742369 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.745313 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.751955 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:41.752805 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:41.753320 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:41.753430 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 81 FOLLOWER]: Starting forced leader election (received explicit request)
W20250111 09:01:41.753473 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:41.753492 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 81 FOLLOWER]: Advancing to term 82
I20250111 09:01:41.754376 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 82 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:41.754776 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 82 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
W20250111 09:01:41.754958 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:41.754976 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 82 candidate_status { last_received { term: 81 index: 808 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:41.755090 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 81 FOLLOWER]: Advancing to term 82
I20250111 09:01:41.755259 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 82 candidate_status { last_received { term: 81 index: 808 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:41.755370 17477 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 81 LEADER]: Stepping down as leader of term 81
I20250111 09:01:41.755414 17477 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 81 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:41.755548 17477 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 808, Committed index: 808, Last appended: 81.808, Last appended by leader: 808, Current term: 81, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:41.755628 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:41.755694 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 81 FOLLOWER]: Advancing to term 82
I20250111 09:01:41.755740 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 83 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:41.755832 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 83 FOLLOWER]: Advancing to term 84
I20250111 09:01:41.756145 17337 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 82 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 82.
I20250111 09:01:41.756378 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 82 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:41.756532 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 84 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:41.756656 17477 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 82 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 82.
I20250111 09:01:41.756847 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 84 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:41.757061 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 84 candidate_status { last_received { term: 82 index: 839 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:41.757154 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 83 FOLLOWER]: Advancing to term 84
I20250111 09:01:41.757258 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 84 candidate_status { last_received { term: 82 index: 839 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:41.757351 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 83 FOLLOWER]: Advancing to term 84
I20250111 09:01:41.757900 17337 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 84 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 84.
I20250111 09:01:41.758018 17477 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 84 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 84.
I20250111 09:01:41.758286 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 84 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:41.758433 18039 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 82 FOLLOWER]: Leader election won for term 82
I20250111 09:01:41.758529 18039 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 82 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:41.758565 18144 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 84 FOLLOWER]: Leader election won for term 84
I20250111 09:01:41.758652 18039 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 808, Committed index: 808, Last appended: 81.808, Last appended by leader: 808, Current term: 82, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:41.758713 18144 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 84 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:41.758841 18144 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 837, Committed index: 837, Last appended: 82.839, Last appended by leader: 839, Current term: 84, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:41.759532 18141 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:41.760308 17178 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 81 to 82, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 82 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:41.760588 17178 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 82 to 84. New cstate: current_term: 84 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:41.762068 17477 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 84 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 82 index: 838. Preceding OpId from leader: term: 84 index: 841. (index mismatch)
W20250111 09:01:41.762369 18141 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:41.762841 18144 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 840, Last known committed idx: 837, Time since last communication: 0.000s
I20250111 09:01:41.763232 17337 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 84 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 82 index: 831. Preceding OpId from leader: term: 84 index: 841. (index mismatch)
I20250111 09:01:41.764009 18144 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 840, Last known committed idx: 828, Time since last communication: 0.000s
W20250111 09:01:41.769078 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.773828 18141 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.774396 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.776082 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.776177 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.780653 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.781863 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.782922 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:41.783557 17337 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 82 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 81 index: 808. Preceding OpId from leader: term: 82 index: 810. (index mismatch)
I20250111 09:01:41.784444 17474 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 82 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 81 index: 808. Preceding OpId from leader: term: 82 index: 810. (index mismatch)
I20250111 09:01:41.785322 18039 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 809, Last known committed idx: 808, Time since last communication: 0.000s
W20250111 09:01:41.790987 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.791466 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:41.791448 18147 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 809, Last known committed idx: 808, Time since last communication: 0.000s
W20250111 09:01:41.795025 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.803356 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.803396 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.808002 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.808329 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.808440 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.808665 18146 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.811215 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.813481 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.814445 18141 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.816713 18123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.822335 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.830472 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.831688 18146 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.837282 18146 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:41.838637 18146 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.025341 18151 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 67 similar messages]
I20250111 09:01:42.201035 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:42.202164 17337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:42.202291 17337 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 82 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:42.202340 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 82 FOLLOWER]: Advancing to term 83
I20250111 09:01:42.203826 17337 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 83 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:42.204053 17339 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 83 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 82. Current term is 83. Ops: [82.928-82.930]
I20250111 09:01:42.204952 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 83 candidate_status { last_received { term: 82 index: 927 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:42.205094 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 82 FOLLOWER]: Advancing to term 83
I20250111 09:01:42.205161 18153 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: INVALID_TERM, Last received: 82.927, Next index: 928, Last known committed idx: 923, Time since last communication: 0.000s
I20250111 09:01:42.205327 18153 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 82 LEADER]: Stepping down as leader of term 82
I20250111 09:01:42.205369 18153 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 82 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:42.205480 18153 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 927, Committed index: 927, Last appended: 82.932, Last appended by leader: 932, Current term: 82, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:42.205564 17337 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 83 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:42.205614 18153 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 82 FOLLOWER]: Advancing to term 83
I20250111 09:01:42.206118 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 83 candidate_status { last_received { term: 82 index: 927 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:42.206789 17614 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 83 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 83 because replica has last-logged OpId of term: 82 index: 932, which is greater than that of the candidate, which has last-logged OpId of term: 82 index: 927.
I20250111 09:01:42.207531 17467 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 83 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 83.
I20250111 09:01:42.208323 17337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:42.208429 17337 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 84 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:42.208479 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 84 FOLLOWER]: Advancing to term 85
I20250111 09:01:42.208487 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 83 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters: b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:42.208709 18134 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 83 FOLLOWER]: Leader election won for term 83
I20250111 09:01:42.208881 18134 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 83 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:42.209023 18134 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 923, Committed index: 923, Last appended: 82.927, Last appended by leader: 927, Current term: 83, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:42.209506 17337 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 85 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:42.209862 17340 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 85 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 84. Current term is 85. Ops: []
I20250111 09:01:42.209985 17337 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 85 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:42.210251 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 85 candidate_status { last_received { term: 84 index: 968 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:42.210322 17614 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 84 LEADER]: Stepping down as leader of term 84
I20250111 09:01:42.210361 17614 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 84 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:42.210459 17614 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 968, Committed index: 968, Last appended: 84.968, Last appended by leader: 968, Current term: 84, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:42.210561 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 84 FOLLOWER]: Advancing to term 85
I20250111 09:01:42.210500 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 82 to 83, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 83 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:42.211210 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 85 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 85.
I20250111 09:01:42.212219 17475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 85 candidate_status { last_received { term: 84 index: 968 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:42.212311 17475 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 84 FOLLOWER]: Advancing to term 85
I20250111 09:01:42.213217 17475 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 85 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 85.
W20250111 09:01:42.213297 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.213335 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:42.213474 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 85 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:42.213613 18134 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 85 FOLLOWER]: Leader election won for term 85
I20250111 09:01:42.213702 18134 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 85 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:42.213829 18134 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 967, Committed index: 967, Last appended: 84.968, Last appended by leader: 968, Current term: 85, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:42.215334 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 84 to 85, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 85 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:42.216749 18072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.218820 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.219448 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.219774 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:42.223385 17614 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 85 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 84 index: 968. Preceding OpId from leader: term: 85 index: 970. (index mismatch)
I20250111 09:01:42.223861 17467 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 85 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 84 index: 967. Preceding OpId from leader: term: 85 index: 970. (index mismatch)
I20250111 09:01:42.224108 18134 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 969, Last known committed idx: 968, Time since last communication: 0.000s
I20250111 09:01:42.225340 18134 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 969, Last known committed idx: 967, Time since last communication: 0.000s
I20250111 09:01:42.225975 17467 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 83 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 82 index: 920. Preceding OpId from leader: term: 83 index: 929. (index mismatch)
I20250111 09:01:42.226310 18173 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 928, Last known committed idx: 915, Time since last communication: 0.000s
I20250111 09:01:42.226715 17614 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 83 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 82 index: 932. Preceding OpId from leader: term: 83 index: 929. (term mismatch)
I20250111 09:01:42.226761 17614 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 928
I20250111 09:01:42.226780 17614 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 82.929
W20250111 09:01:42.226835 17614 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Aborted: Op aborted by new leader
I20250111 09:01:42.226938 17614 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 82.930
W20250111 09:01:42.226998 17614 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:01:42.227104 17614 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 82.931
W20250111 09:01:42.227169 17614 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:01:42.227265 17614 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 82.932
W20250111 09:01:42.227312 17614 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Aborted: Op aborted by new leader
I20250111 09:01:42.231412 18151 mvcc.cc:204] Tried to move back new op lower bound from 7113056674706247680 to 7113056674669092864. Current Snapshot: MvccSnapshot[applied={T|T < 7113056674702573568}]
I20250111 09:01:42.232846 18171 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 928, Last known committed idx: 927, Time since last communication: 0.000s
I20250111 09:01:42.232944 18171 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 82.928 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 927
I20250111 09:01:42.233465 17611 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 927
I20250111 09:01:42.233505 17611 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 82.928
W20250111 09:01:42.233564 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
W20250111 09:01:42.277243 18167 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 70 similar messages]
I20250111 09:01:42.314430 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:42.315096 17474 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:42.315340 17474 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 83 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:42.315423 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 83 FOLLOWER]: Advancing to term 84
I20250111 09:01:42.316452 17474 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 84 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:42.316795 17474 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 84 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:42.317319 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 84 candidate_status { last_received { term: 83 index: 955 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:42.317432 17339 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 83 LEADER]: Stepping down as leader of term 83
I20250111 09:01:42.317483 17339 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 83 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:42.318405 17477 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 84 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 83. Current term is 84. Ops: [83.956-83.958]
I20250111 09:01:42.318542 17339 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 955, Committed index: 955, Last appended: 83.958, Last appended by leader: 958, Current term: 83, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:42.318681 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 83 FOLLOWER]: Advancing to term 84
I20250111 09:01:42.319031 17475 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:42.319267 17475 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 85 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:42.319386 17475 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 85 FOLLOWER]: Advancing to term 86
I20250111 09:01:42.320665 17339 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 84 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 84 because replica has last-logged OpId of term: 83 index: 958, which is greater than that of the candidate, which has last-logged OpId of term: 83 index: 955.
W20250111 09:01:42.321030 18137 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: Replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 84 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
I20250111 09:01:42.321542 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 84 candidate_status { last_received { term: 83 index: 955 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:42.321650 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 83 FOLLOWER]: Advancing to term 84
W20250111 09:01:42.322438 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:42.322633 17614 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 84 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 84 because replica has last-logged OpId of term: 83 index: 958, which is greater than that of the candidate, which has last-logged OpId of term: 83 index: 955.
I20250111 09:01:42.322855 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 84 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:42.322993 18127 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 84 FOLLOWER]: Leader election lost for term 84. Reason: could not achieve majority
I20250111 09:01:42.323132 17475 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 86 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:42.323657 17477 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 86 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 85. Current term is 86. Ops: [85.989-85.989]
I20250111 09:01:42.323642 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 86 candidate_status { last_received { term: 85 index: 988 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:42.323776 17337 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 85 LEADER]: Stepping down as leader of term 85
I20250111 09:01:42.323848 17337 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 85 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
W20250111 09:01:42.323920 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:42.323951 17475 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 86 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:42.324018 17337 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 988, Committed index: 988, Last appended: 85.992, Last appended by leader: 992, Current term: 85, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:42.324121 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 85 FOLLOWER]: Advancing to term 86
I20250111 09:01:42.324436 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 86 candidate_status { last_received { term: 85 index: 988 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:42.325055 17337 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 86 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 86 because replica has last-logged OpId of term: 85 index: 992, which is greater than that of the candidate, which has last-logged OpId of term: 85 index: 988.
I20250111 09:01:42.325337 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 86 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:42.325538 18127 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 86 FOLLOWER]: Leader election lost for term 86. Reason: could not achieve majority
W20250111 09:01:42.328927 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.330422 18064 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 77 similar messages]
W20250111 09:01:42.330468 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.333444 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.334082 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.335523 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.336781 18064 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.337723 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.341192 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.342527 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.342912 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.343552 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.347167 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.348635 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.349088 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.349627 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.355350 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.358217 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.358445 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.359191 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.362149 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.367033 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.369060 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.369339 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.369978 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.377422 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.380107 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.380124 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.381171 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.388916 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.390761 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.391799 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.394120 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.400688 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.404656 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.406319 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.407059 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.414723 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.417735 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.421468 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.421924 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.429327 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.431641 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.437319 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.437659 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.447121 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.449177 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.452811 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.453752 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.463457 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.465389 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.468446 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.472144 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.480382 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.483912 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.485458 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.488055 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.496688 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.501312 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.503734 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.508056 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.515445 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.519549 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.521585 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.527287 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.536450 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.540467 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.544019 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.546180 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.555091 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.560798 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.565315 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.570283 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.577418 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.583703 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.586179 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.594338 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.602434 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.608733 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.610669 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.620440 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.626134 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.633848 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.635449 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.648216 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.652580 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.659690 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.663375 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.676447 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.677475 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.683725 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.690779 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.701704 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.703156 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.711876 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.716573 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.730741 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.731850 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.741776 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.744367 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.759697 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.761587 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.769840 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.777184 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.787712 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.792297 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.797907 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.807132 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.817507 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.825690 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.829787 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.839968 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.851687 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.858574 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.859984 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.873586 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.883940 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.893644 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.894869 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.909518 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.919463 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.929812 18110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.929888 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.944247 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.953647 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.963864 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.966114 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.982826 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:42.988006 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:42.999720 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:43.002244 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:43.021684 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.025048 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.037858 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.039039 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.058446 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:43.062745 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.075557 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.078220 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.098158 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.103993 18168 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 59 similar messages]
W20250111 09:01:43.104043 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:43.114836 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:43.118374 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:43.136099 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.146183 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.158562 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.160490 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.176090 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:43.187517 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.197458 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.201583 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.216689 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.228318 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:43.242209 18168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:43.242250 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:43.260576 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.272802 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.286574 18063 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 47 similar messages]
W20250111 09:01:43.286664 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:43.288499 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
W20250111 09:01:43.288522 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:43.288972 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:43.289129 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 84 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:43.289206 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 84 FOLLOWER]: Advancing to term 85
I20250111 09:01:43.289980 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 85 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:43.290313 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 85 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:43.290627 17475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 85 candidate_status { last_received { term: 83 index: 958 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:43.290750 17475 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 84 FOLLOWER]: Advancing to term 85
I20250111 09:01:43.290750 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 85 candidate_status { last_received { term: 83 index: 958 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:43.290848 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 84 FOLLOWER]: Advancing to term 85
I20250111 09:01:43.291725 17339 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 85 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 85.
I20250111 09:01:43.291725 17475 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 85 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 85.
I20250111 09:01:43.292042 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 85 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:43.292240 18145 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 85 FOLLOWER]: Leader election won for term 85
I20250111 09:01:43.292266 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:43.292378 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 85 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:43.292419 18145 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 85 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:43.292447 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 85 FOLLOWER]: Advancing to term 86
I20250111 09:01:43.292608 18145 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 955, Committed index: 955, Last appended: 83.958, Last appended by leader: 958, Current term: 85, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:43.293190 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 86 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:43.293493 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 86 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:43.293798 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 86 candidate_status { last_received { term: 85 index: 992 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:43.294481 17339 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 86 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 86.
I20250111 09:01:43.294692 17475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 86 candidate_status { last_received { term: 85 index: 992 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:43.294781 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 86 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:43.294847 17475 raft_consensus.cc:2388] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 86 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c in current term 86: Already voted for candidate 262623ac6df745c785ce267ea39bd265 in this term.
I20250111 09:01:43.294970 18145 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 86 FOLLOWER]: Leader election won for term 86
I20250111 09:01:43.295073 18145 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 86 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:43.295205 18145 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 988, Committed index: 988, Last appended: 85.992, Last appended by leader: 992, Current term: 86, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:43.295389 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 83 to 85, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 85 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:43.298576 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 85 to 86, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 86 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:43.303115 17475 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 85 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 83 index: 955. Preceding OpId from leader: term: 85 index: 960. (index mismatch)
I20250111 09:01:43.303313 17339 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 85 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 83 index: 958. Preceding OpId from leader: term: 85 index: 960. (index mismatch)
I20250111 09:01:43.303488 18145 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 959, Last known committed idx: 955, Time since last communication: 0.000s
I20250111 09:01:43.303709 18176 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 959, Last known committed idx: 955, Time since last communication: 0.000s
I20250111 09:01:43.308794 18190 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39814 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=1216, attempt_no=0}) took 1006 ms. Trace:
I20250111 09:01:43.308979 18191 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39814 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=1217, attempt_no=0}) took 1003 ms. Trace:
I20250111 09:01:43.308851 18190 rpcz_store.cc:276] 0111 09:01:42.302059 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:42.302095 (+ 36us) service_pool.cc:225] Handling call
0111 09:01:43.308782 (+1006687us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:42.305256 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 2a4e17d0e2894eaa95b650f1af0ce650
0111 09:01:42.306027 (+ 771us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:42.306029 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:01:42.306030 (+ 1us) tablet.cc:659] Decoding operations
0111 09:01:42.306069 (+ 39us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:01:42.306096 (+ 27us) tablet.cc:698] Row locks acquired
0111 09:01:42.306097 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:01:42.306121 (+ 24us) write_op.cc:270] Start()
0111 09:01:42.306138 (+ 17us) write_op.cc:276] Timestamp: P: 1736586102306061 usec, L: 0
0111 09:01:42.306139 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:01:42.306229 (+ 90us) log.cc:844] Serialized 306 byte log entry
0111 09:01:43.306999 (+1000770us) op_driver.cc:464] REPLICATION: finished
0111 09:01:43.308467 (+ 1468us) write_op.cc:301] APPLY: starting
0111 09:01:43.308484 (+ 17us) tablet.cc:1364] starting BulkCheckPresence
0111 09:01:43.308538 (+ 54us) tablet.cc:1367] finished BulkCheckPresence
0111 09:01:43.308539 (+ 1us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:01:43.308572 (+ 33us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:01:43.308609 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=25
0111 09:01:43.308614 (+ 5us) write_op.cc:312] APPLY: finished
0111 09:01:43.308676 (+ 62us) log.cc:844] Serialized 26 byte log entry
0111 09:01:43.308721 (+ 45us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:43.308730 (+ 9us) write_op.cc:454] Released schema lock
0111 09:01:43.308734 (+ 4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1429,"num_ops":1,"prepare.queue_time_us":2907,"prepare.run_cpu_time_us":305,"prepare.run_wall_time_us":1020,"replication_time_us":1000841,"thread_start_us":50,"threads_started":1}]]}
I20250111 09:01:43.309037 18191 rpcz_store.cc:276] 0111 09:01:42.305250 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:42.305280 (+ 30us) service_pool.cc:225] Handling call
0111 09:01:43.308971 (+1003691us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:42.306297 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 2a4e17d0e2894eaa95b650f1af0ce650
0111 09:01:42.306346 (+ 49us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:42.306347 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:01:42.306348 (+ 1us) tablet.cc:659] Decoding operations
0111 09:01:42.306380 (+ 32us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:01:42.306401 (+ 21us) tablet.cc:698] Row locks acquired
0111 09:01:42.306402 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:01:42.306422 (+ 20us) write_op.cc:270] Start()
0111 09:01:42.306436 (+ 14us) write_op.cc:276] Timestamp: P: 1736586102306363 usec, L: 0
0111 09:01:42.306438 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:01:42.306551 (+ 113us) log.cc:844] Serialized 306 byte log entry
0111 09:01:43.307094 (+1000543us) op_driver.cc:464] REPLICATION: finished
0111 09:01:43.308726 (+ 1632us) write_op.cc:301] APPLY: starting
0111 09:01:43.308737 (+ 11us) tablet.cc:1364] starting BulkCheckPresence
0111 09:01:43.308785 (+ 48us) tablet.cc:1367] finished BulkCheckPresence
0111 09:01:43.308786 (+ 1us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:01:43.308818 (+ 32us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:01:43.308846 (+ 28us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=25
0111 09:01:43.308852 (+ 6us) write_op.cc:312] APPLY: finished
0111 09:01:43.308886 (+ 34us) log.cc:844] Serialized 26 byte log entry
0111 09:01:43.308919 (+ 33us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:43.308926 (+ 7us) write_op.cc:454] Released schema lock
0111 09:01:43.308930 (+ 4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1603,"num_ops":1,"prepare.queue_time_us":784,"prepare.run_cpu_time_us":302,"prepare.run_wall_time_us":301,"replication_time_us":1000644,"spinlock_wait_cycles":8576,"thread_start_us":35,"threads_started":1}]]}
W20250111 09:01:43.310339 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.311215 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.317271 18174 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.317360 18199 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.318554 18200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:43.325263 17339 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 86 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 85 index: 992. Preceding OpId from leader: term: 86 index: 994. (index mismatch)
I20250111 09:01:43.325259 17475 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 86 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 85 index: 988. Preceding OpId from leader: term: 86 index: 994. (index mismatch)
I20250111 09:01:43.325667 18145 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 993, Last known committed idx: 987, Time since last communication: 0.000s
I20250111 09:01:43.325856 18176 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 993, Last known committed idx: 988, Time since last communication: 0.000s
W20250111 09:01:43.332864 18200 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 41 similar messages]
W20250111 09:01:43.332973 18200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:43.333235 18196 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.ParticipateInTransaction from 127.14.61.3:38369 (ReqId={client: c602c61032fa436092728430b0406df9, seq_no=998, attempt_no=0}) took 1016 ms. Trace:
I20250111 09:01:43.333307 18196 rpcz_store.cc:276] 0111 09:01:42.316929 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:42.316964 (+ 35us) service_pool.cc:225] Handling call
0111 09:01:43.333222 (+1016258us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:42.318180 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:42.318206 (+ 26us) participant_op.cc:199] PREPARE: finished
0111 09:01:42.318222 (+ 16us) participant_op.cc:212] START. Timestamp: 1736586102318160
0111 09:01:42.318223 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:01:42.318288 (+ 65us) log.cc:844] Serialized 67 byte log entry
0111 09:01:43.330177 (+1011889us) op_driver.cc:464] REPLICATION: finished
0111 09:01:43.333083 (+ 2906us) participant_op.cc:265] APPLY: starting
0111 09:01:43.333160 (+ 77us) participant_op.cc:270] APPLY: finished
0111 09:01:43.333183 (+ 23us) log.cc:844] Serialized 15 byte log entry
0111 09:01:43.333202 (+ 19us) participant_op.cc:94] Released txn lock
0111 09:01:43.333209 (+ 7us) participant_op.cc:291] FINISH: Op applied
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2855,"prepare.queue_time_us":1065,"prepare.run_cpu_time_us":139,"prepare.run_wall_time_us":139,"replication_time_us":1011935}]]}
W20250111 09:01:43.333762 18192 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:43.333860 18192 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39814 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=1218, attempt_no=0}) took 1028 ms. Trace:
I20250111 09:01:43.334111 18192 rpcz_store.cc:276] 0111 09:01:42.305319 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:42.305391 (+ 72us) service_pool.cc:225] Handling call
0111 09:01:43.333853 (+1028462us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:42.319405 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:42.319424 (+ 19us) participant_op.cc:199] PREPARE: finished
0111 09:01:42.319442 (+ 18us) participant_op.cc:212] START. Timestamp: 1736586102319380
0111 09:01:42.319443 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:01:42.319513 (+ 70us) log.cc:844] Serialized 67 byte log entry
0111 09:01:43.330219 (+1010706us) op_driver.cc:464] REPLICATION: finished
0111 09:01:43.333132 (+ 2913us) participant_op.cc:265] APPLY: starting
0111 09:01:43.333598 (+ 466us) participant_op.cc:270] APPLY: finished
0111 09:01:43.333622 (+ 24us) log.cc:844] Serialized 15 byte log entry
0111 09:01:43.333637 (+ 15us) participant_op.cc:94] Released txn lock
0111 09:01:43.333642 (+ 5us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:01:43.333744 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER)
0111 09:01:43.333749 (+ 5us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:43.333751 (+ 2us) write_op.cc:454] Released schema lock
0111 09:01:43.333754 (+ 3us) write_op.cc:335] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":36,"txn-participant-registration.run_cpu_time_us":295,"txn-participant-registration.run_wall_time_us":12562,"child_traces":[["op",{"apply.queue_time_us":2879,"prepare.queue_time_us":1261,"prepare.run_cpu_time_us":143,"prepare.run_wall_time_us":339,"replication_time_us":1010766,"child_traces":[["op",{}]]}]]}
I20250111 09:01:43.334822 18192 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39814 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=1221, attempt_no=0}) took 1015 ms. Trace:
I20250111 09:01:43.334878 18192 rpcz_store.cc:276] 0111 09:01:42.319345 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:42.319397 (+ 52us) service_pool.cc:225] Handling call
0111 09:01:43.334813 (+1015416us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:42.319896 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 6ab63c6543ad4a0ab83e76267a725edd
0111 09:01:42.319950 (+ 54us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:42.319951 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:01:42.319952 (+ 1us) tablet.cc:659] Decoding operations
0111 09:01:42.319987 (+ 35us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:01:42.320010 (+ 23us) tablet.cc:698] Row locks acquired
0111 09:01:42.320011 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:01:42.320032 (+ 21us) write_op.cc:270] Start()
0111 09:01:42.320046 (+ 14us) write_op.cc:276] Timestamp: P: 1736586102319972 usec, L: 0
0111 09:01:42.320047 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:01:42.320120 (+ 73us) log.cc:844] Serialized 306 byte log entry
0111 09:01:43.330287 (+1010167us) op_driver.cc:464] REPLICATION: finished
0111 09:01:43.334468 (+ 4181us) write_op.cc:301] APPLY: starting
0111 09:01:43.334482 (+ 14us) tablet.cc:1364] starting BulkCheckPresence
0111 09:01:43.334541 (+ 59us) tablet.cc:1367] finished BulkCheckPresence
0111 09:01:43.334542 (+ 1us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:01:43.334609 (+ 67us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:01:43.334640 (+ 31us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=25
0111 09:01:43.334648 (+ 8us) write_op.cc:312] APPLY: finished
0111 09:01:43.334685 (+ 37us) log.cc:844] Serialized 26 byte log entry
0111 09:01:43.334723 (+ 38us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:43.334732 (+ 9us) write_op.cc:454] Released schema lock
0111 09:01:43.334737 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4156,"num_ops":1,"prepare.queue_time_us":248,"prepare.run_cpu_time_us":267,"prepare.run_wall_time_us":267,"replication_time_us":1010226,"spinlock_wait_cycles":70016}]]}
W20250111 09:01:43.335644 18191 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:43.335827 18191 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39814 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=1219, attempt_no=0}) took 1026 ms. Trace:
I20250111 09:01:43.335942 18191 rpcz_store.cc:276] 0111 09:01:42.309729 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:42.309762 (+ 33us) service_pool.cc:225] Handling call
0111 09:01:43.335816 (+1026054us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:42.319761 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:42.319775 (+ 14us) participant_op.cc:199] PREPARE: finished
0111 09:01:42.319799 (+ 24us) participant_op.cc:212] START. Timestamp: 1736586102319739
0111 09:01:42.319800 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:01:42.319845 (+ 45us) log.cc:844] Serialized 67 byte log entry
0111 09:01:43.330247 (+1010402us) op_driver.cc:464] REPLICATION: finished
0111 09:01:43.334463 (+ 4216us) participant_op.cc:265] APPLY: starting
0111 09:01:43.335016 (+ 553us) participant_op.cc:270] APPLY: finished
0111 09:01:43.335054 (+ 38us) log.cc:844] Serialized 15 byte log entry
0111 09:01:43.335267 (+ 213us) participant_op.cc:94] Released txn lock
0111 09:01:43.335273 (+ 6us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:01:43.335624 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER)
0111 09:01:43.335630 (+ 6us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:43.335634 (+ 4us) write_op.cc:454] Released schema lock
0111 09:01:43.335636 (+ 2us) write_op.cc:335] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":39,"txn-participant-registration.run_cpu_time_us":236,"txn-participant-registration.run_wall_time_us":8324,"child_traces":[["op",{"apply.queue_time_us":3814,"prepare.queue_time_us":1504,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":113,"replication_time_us":1010439,"child_traces":[["op",{}]]}]]}
W20250111 09:01:43.337965 18200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.338688 18199 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.344152 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.344206 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.348331 18063 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.367486 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:43.369786 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.113050 18206 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 76 similar messages]
I20250111 09:01:44.156925 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:44.157593 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:44.158421 17338 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 85 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:44.158558 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 85 FOLLOWER]: Advancing to term 86
I20250111 09:01:44.160256 17338 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 86 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:44.160575 17338 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 86 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:44.160907 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 86 candidate_status { last_received { term: 85 index: 1187 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:44.161041 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 85 FOLLOWER]: Advancing to term 86
I20250111 09:01:44.161219 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:44.161962 17474 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 86 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 86.
I20250111 09:01:44.162534 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 86 candidate_status { last_received { term: 85 index: 1187 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:44.163049 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 86 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:44.164041 17475 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 86 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 85. Current term is 86. Ops: [85.1188-85.1188]
I20250111 09:01:44.164155 17340 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 86 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:44.164273 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 86 FOLLOWER]: Advancing to term 87
I20250111 09:01:44.164573 18216 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 86 FOLLOWER]: Leader election won for term 86
I20250111 09:01:44.164886 18216 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 86 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:44.165051 18215 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: INVALID_TERM, Last received: 85.1187, Next index: 1188, Last known committed idx: 1187, Time since last communication: 0.000s
I20250111 09:01:44.165328 18204 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 85 LEADER]: Stepping down as leader of term 85
I20250111 09:01:44.165395 18204 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 85 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:44.165542 18204 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1187, Committed index: 1187, Last appended: 85.1188, Last appended by leader: 1188, Current term: 85, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:44.165696 18204 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 85 FOLLOWER]: Advancing to term 86
I20250111 09:01:44.167008 18216 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1187, Committed index: 1187, Last appended: 85.1187, Last appended by leader: 1187, Current term: 86, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:44.167691 17336 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 86 LEADER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 85. Current term is 86. Ops: [85.1188-85.1188]
I20250111 09:01:44.167802 17340 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 87 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:44.168524 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 85 to 86, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 86 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:44.170321 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:44.170467 17339 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 87 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 86. Current term is 87. Ops: [86.1226-86.1226]
I20250111 09:01:44.171005 18215 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: INVALID_TERM, Last received: 86.1225, Next index: 1226, Last known committed idx: 1221, Time since last communication: 0.000s
I20250111 09:01:44.171397 18215 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 86 LEADER]: Stepping down as leader of term 86
I20250111 09:01:44.171485 18215 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 86 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:44.171828 18215 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1225, Committed index: 1225, Last appended: 86.1229, Last appended by leader: 1229, Current term: 86, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:44.172000 18215 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 86 FOLLOWER]: Advancing to term 87
I20250111 09:01:44.173725 17474 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 86 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 85 index: 1187. Preceding OpId from leader: term: 86 index: 1189. (index mismatch)
I20250111 09:01:44.174021 18216 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1188, Last known committed idx: 1187, Time since last communication: 0.000s
I20250111 09:01:44.174440 17614 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 86 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 85 index: 1188. Preceding OpId from leader: term: 86 index: 1189. (index mismatch)
I20250111 09:01:44.174804 17340 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 87 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:44.175455 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 87 candidate_status { last_received { term: 86 index: 1225 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:44.175592 17614 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 87 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 87 because replica has last-logged OpId of term: 86 index: 1229, which is greater than that of the candidate, which has last-logged OpId of term: 86 index: 1225.
I20250111 09:01:44.175742 18216 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1188, Last known committed idx: 1187, Time since last communication: 0.000s
I20250111 09:01:44.176123 18216 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 85.1188 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1187
I20250111 09:01:44.176818 17614 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 1187
I20250111 09:01:44.176865 17614 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 85.1188
I20250111 09:01:44.176846 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 87 candidate_status { last_received { term: 86 index: 1225 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:44.176925 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 86 FOLLOWER]: Advancing to term 87
W20250111 09:01:44.176935 17614 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Aborted: Op aborted by new leader
W20250111 09:01:44.177572 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:44.178282 17474 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 87 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 87 because replica has last-logged OpId of term: 86 index: 1226, which is greater than that of the candidate, which has last-logged OpId of term: 86 index: 1225.
I20250111 09:01:44.178493 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 87 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:44.178659 18216 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 87 FOLLOWER]: Leader election lost for term 87. Reason: could not achieve majority
W20250111 09:01:44.179368 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.179458 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.179870 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.181785 18210 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.187139 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.187361 18222 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.190102 18222 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.191286 18229 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.191797 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.193652 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.194907 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.197141 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.198555 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.201941 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.202539 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.204309 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.206086 18231 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.206722 18232 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.210130 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.211587 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.213078 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.213249 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.216171 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.216747 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.218261 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.218976 18232 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.219897 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.220062 18231 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.223484 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.224150 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.224169 18231 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.228797 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.229115 18231 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.230767 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.232971 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.233533 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.236184 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.237524 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.238867 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.239768 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.243408 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.245359 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.249202 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.249689 18231 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.249822 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.249852 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.253391 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.256954 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.261729 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.263255 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.263947 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.264206 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.264441 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.267114 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.274668 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.275445 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.277034 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.278213 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.279043 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.280462 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.288885 18228 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 30 similar messages]
W20250111 09:01:44.288945 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.289937 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.291576 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.291997 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.292102 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.294627 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.302968 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.304065 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.307034 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.307660 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.309645 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.312322 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.318084 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.321722 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.322091 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.322595 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.326191 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.329767 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.333848 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.335278 18230 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 25 similar messages]
W20250111 09:01:44.335340 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.338899 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.340283 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.341782 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.347425 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.350467 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.353776 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.358363 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.360639 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.360667 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.367841 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.368798 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.368960 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.376992 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.379143 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.380381 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.387223 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.388599 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.391099 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.398638 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.398748 18231 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.399894 18231 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.409368 18231 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.409546 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.413167 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.418326 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.418381 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.420536 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.430804 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.433784 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.436648 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.439107 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.441223 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.444087 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.450006 18231 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.458709 18231 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.460043 18231 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.464596 18231 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.464653 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.469437 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.470273 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.482306 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.482381 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.487257 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.488483 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.493536 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.494834 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.506073 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.506318 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.512619 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.513233 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.517653 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.518366 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.529960 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.534134 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.537413 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.540431 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.541572 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.545605 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.557677 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.558774 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.566071 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.568126 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.568244 18231 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.569464 18231 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.584120 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.584942 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.592620 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.593932 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.598382 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.598644 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.613148 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.613164 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.624370 18227 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.625545 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.625567 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.625576 18229 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.643491 18231 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.644483 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.653113 18231 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.653175 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.654001 18232 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.658100 18232 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.674829 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.675179 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.682965 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.684733 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.685230 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.688530 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.708420 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.709419 18229 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.717607 18229 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.717957 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.718211 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.719242 18229 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.743341 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.743399 18232 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.747977 18232 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.749284 18232 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.750476 18232 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.753794 18232 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.779670 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.779754 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.783567 18229 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.785579 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.786507 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.787797 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.813797 18229 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.813871 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.819912 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.820586 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.824448 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.824465 18229 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.849601 18232 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.851648 18232 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.856436 18232 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.857739 18232 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.862149 18232 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.862341 18230 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.886801 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.889680 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.892721 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.892789 18229 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.900334 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.901760 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:44.923529 18229 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:44.927007 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:44.927476 17474 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:44.927603 17474 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 86 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:44.927671 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 86 FOLLOWER]: Advancing to term 87
I20250111 09:01:44.928694 17474 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 87 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:44.928799 18229 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:44.929440 17474 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 87 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:44.929728 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 87 candidate_status { last_received { term: 86 index: 1195 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:44.929837 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 86 FOLLOWER]: Advancing to term 87
I20250111 09:01:44.930037 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 87 candidate_status { last_received { term: 86 index: 1195 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
W20250111 09:01:44.930089 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:44.930145 17337 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 86 LEADER]: Stepping down as leader of term 86
I20250111 09:01:44.930187 17337 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 86 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:44.930326 17337 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1195, Committed index: 1195, Last appended: 86.1195, Last appended by leader: 1195, Current term: 86, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:44.930471 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 86 FOLLOWER]: Advancing to term 87
I20250111 09:01:44.930451 17474 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:44.930547 17474 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 87 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:44.930629 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 87 FOLLOWER]: Advancing to term 88
I20250111 09:01:44.930903 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 87 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 87.
I20250111 09:01:44.931177 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 87 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:44.931360 17474 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 88 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:44.931555 17337 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 87 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 87.
I20250111 09:01:44.931586 18233 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 87 FOLLOWER]: Leader election won for term 87
I20250111 09:01:44.931676 17474 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 88 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:44.931726 18233 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 87 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:44.931874 18233 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1195, Committed index: 1195, Last appended: 86.1195, Last appended by leader: 1195, Current term: 87, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:44.931973 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 88 candidate_status { last_received { term: 86 index: 1226 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:44.932096 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 87 FOLLOWER]: Advancing to term 88
I20250111 09:01:44.932368 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 88 candidate_status { last_received { term: 86 index: 1226 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:44.932463 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 87 FOLLOWER]: Advancing to term 88
I20250111 09:01:44.932911 17614 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 88 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 88 because replica has last-logged OpId of term: 86 index: 1229, which is greater than that of the candidate, which has last-logged OpId of term: 86 index: 1226.
I20250111 09:01:44.933311 17337 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 88 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 88.
I20250111 09:01:44.933672 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 88 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters: b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:44.933472 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 86 to 87, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 87 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:44.933969 18234 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 88 FOLLOWER]: Leader election won for term 88
I20250111 09:01:44.934063 18234 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 88 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:44.934190 18234 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1225, Committed index: 1225, Last appended: 86.1226, Last appended by leader: 1226, Current term: 88, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:44.934239 18175 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:44.936744 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 86 to 88, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 88 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:44.937211 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.938659 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:44.964655 17337 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 88 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 86 index: 1225. Preceding OpId from leader: term: 88 index: 1228. (index mismatch)
I20250111 09:01:44.964658 17614 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 88 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 86 index: 1229. Preceding OpId from leader: term: 88 index: 1228. (term mismatch)
I20250111 09:01:44.964705 17614 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 1227
I20250111 09:01:44.964721 17614 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 86.1228
W20250111 09:01:44.964810 17614 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:01:44.965057 18233 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1227, Last known committed idx: 1221, Time since last communication: 0.000s
I20250111 09:01:44.965117 17614 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 86.1229
W20250111 09:01:44.965222 17614 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:01:44.965726 18233 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1227, Last known committed idx: 1225, Time since last communication: 0.000s
I20250111 09:01:44.965828 18233 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 86.1227 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1225
I20250111 09:01:44.966425 17614 raft_consensus.cc:1212] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 88 FOLLOWER]: Deduplicated request from leader. Original: 86.1225->[86.1226-88.1228] Dedup: 86.1226->[88.1227-88.1228]
I20250111 09:01:44.966508 17614 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 1226
I20250111 09:01:44.966544 17614 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 86.1227
W20250111 09:01:44.966637 17614 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Aborted: Op aborted by new leader
W20250111 09:01:44.973274 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:44.974339 18228 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:44.979794 17614 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 87 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 86 index: 1195. Preceding OpId from leader: term: 87 index: 1197. (index mismatch)
I20250111 09:01:44.980216 18234 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1196, Last known committed idx: 1195, Time since last communication: 0.000s
I20250111 09:01:44.980544 17340 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 87 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 86 index: 1195. Preceding OpId from leader: term: 87 index: 1197. (index mismatch)
I20250111 09:01:44.982403 18233 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1196, Last known committed idx: 1195, Time since last communication: 0.000s
W20250111 09:01:45.029297 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:45.350145 18265 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 57 similar messages]
I20250111 09:01:45.600564 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:45.601048 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:45.601172 17613 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 87 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:45.601228 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 87 FOLLOWER]: Advancing to term 88
I20250111 09:01:45.602128 17613 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 88 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:45.602886 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 88 candidate_status { last_received { term: 87 index: 1355 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:45.603017 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 87 FOLLOWER]: Advancing to term 88
I20250111 09:01:45.602540 17613 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 88 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:45.604139 17339 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 88 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 88.
I20250111 09:01:45.604350 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:45.604454 17613 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 88 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:45.604503 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 88 FOLLOWER]: Advancing to term 89
I20250111 09:01:45.605520 17613 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 89 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:45.605734 17339 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 88 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 87. Current term is 88. Ops: [87.1351-87.1355]
I20250111 09:01:45.605823 17613 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 89 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:45.604350 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 88 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:45.606210 18266 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 88 FOLLOWER]: Leader election won for term 88
I20250111 09:01:45.606292 18266 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 88 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:45.606397 18266 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1353, Committed index: 1353, Last appended: 87.1355, Last appended by leader: 1355, Current term: 88, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:45.607064 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 89 candidate_status { last_received { term: 88 index: 1408 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:45.607151 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 88 FOLLOWER]: Advancing to term 89
I20250111 09:01:45.607957 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 87 to 88, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 88 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:45.608337 18234 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: INVALID_TERM, Last received: 87.1350, Next index: 1351, Last known committed idx: 1348, Time since last communication: 0.000s
I20250111 09:01:45.608511 17339 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 89 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 89.
I20250111 09:01:45.608611 18250 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 87 LEADER]: Stepping down as leader of term 87
I20250111 09:01:45.608665 18250 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 87 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:45.608790 18250 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1355, Committed index: 1355, Last appended: 87.1356, Last appended by leader: 1356, Current term: 87, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:45.608842 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 88 candidate_status { last_received { term: 87 index: 1355 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:45.608939 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 89 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:45.609217 18145 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 89 FOLLOWER]: Leader election won for term 89
I20250111 09:01:45.609292 18145 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 89 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:45.609269 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 89 candidate_status { last_received { term: 88 index: 1408 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:45.609349 17474 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 88 LEADER]: Stepping down as leader of term 88
I20250111 09:01:45.609377 17474 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 88 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:45.609387 18145 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1406, Committed index: 1406, Last appended: 88.1408, Last appended by leader: 1408, Current term: 89, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:45.609474 17474 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1406, Committed index: 1406, Last appended: 88.1409, Last appended by leader: 1409, Current term: 88, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:45.609573 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 88 FOLLOWER]: Advancing to term 89
W20250111 09:01:45.609807 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:45.610414 18250 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 87 FOLLOWER]: Advancing to term 88
I20250111 09:01:45.610457 17474 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 89 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 89 because replica has last-logged OpId of term: 88 index: 1409, which is greater than that of the candidate, which has last-logged OpId of term: 88 index: 1408.
W20250111 09:01:45.611580 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:45.613534 17477 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 88 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 88 because replica has last-logged OpId of term: 87 index: 1356, which is greater than that of the candidate, which has last-logged OpId of term: 87 index: 1355.
I20250111 09:01:45.613732 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 88 to 89, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 89 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
W20250111 09:01:45.614673 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:45.615758 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:45.615794 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:45.618142 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:45.618398 18232 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:45.619791 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:45.620496 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:45.621217 18273 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 54 similar messages]
W20250111 09:01:45.621459 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:45.621490 18273 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:45.623620 18231 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:45.626756 17337 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 88 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 87 index: 1350. Preceding OpId from leader: term: 88 index: 1357. (index mismatch)
I20250111 09:01:45.627188 18262 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1356, Last known committed idx: 1348, Time since last communication: 0.000s
I20250111 09:01:45.627697 17477 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 89 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 88 index: 1409. Preceding OpId from leader: term: 89 index: 1410. (index mismatch)
I20250111 09:01:45.627892 17339 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 89 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 88 index: 1407. Preceding OpId from leader: term: 89 index: 1410. (index mismatch)
I20250111 09:01:45.627916 17474 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 88 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 87 index: 1356. Preceding OpId from leader: term: 88 index: 1357. (index mismatch)
I20250111 09:01:45.628484 18262 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1409, Last known committed idx: 1405, Time since last communication: 0.000s
W20250111 09:01:45.628548 18274 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:45.629763 18262 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1409, Last known committed idx: 1406, Time since last communication: 0.000s
I20250111 09:01:45.629853 18262 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer 262623ac6df745c785ce267ea39bd265 log is divergent from this leader: its last log entry 88.1409 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1406
I20250111 09:01:45.630102 18264 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1356, Last known committed idx: 1355, Time since last communication: 0.000s
I20250111 09:01:45.630205 18264 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer 262623ac6df745c785ce267ea39bd265 log is divergent from this leader: its last log entry 87.1356 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1355
I20250111 09:01:45.630820 17474 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 1355
I20250111 09:01:45.630865 17474 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 87.1356
W20250111 09:01:45.630955 17474 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
W20250111 09:01:45.631076 18274 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:45.632270 17474 raft_consensus.cc:1212] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 89 FOLLOWER]: Deduplicated request from leader. Original: 88.1406->[88.1407-89.1411] Dedup: 88.1408->[89.1409-89.1411]
I20250111 09:01:45.632339 17474 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 1408
I20250111 09:01:45.632361 17474 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 88.1409
W20250111 09:01:45.632432 17474 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
W20250111 09:01:45.633239 18273 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:45.636008 18254 mvcc.cc:204] Tried to move back new op lower bound from 7113056688642281472 to 7113056688565018624. Current Snapshot: MvccSnapshot[applied={T|T < 7113056688642281472}]
W20250111 09:01:45.658699 18261 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 37 similar messages]
I20250111 09:01:46.549180 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:46.553169 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:46.553567 17340 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 88 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:46.553716 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 88 FOLLOWER]: Advancing to term 89
I20250111 09:01:46.555601 17340 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 89 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:46.556514 17340 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 89 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:46.557426 17337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:46.557590 17337 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 89 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:46.557694 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 89 FOLLOWER]: Advancing to term 90
I20250111 09:01:46.558553 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 89 candidate_status { last_received { term: 88 index: 1604 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:46.557842 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 89 candidate_status { last_received { term: 88 index: 1604 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:46.558658 17613 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 88 LEADER]: Stepping down as leader of term 88
I20250111 09:01:46.558696 17613 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 88 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:46.558807 17613 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1604, Committed index: 1604, Last appended: 88.1608, Last appended by leader: 1608, Current term: 88, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:46.558887 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 88 FOLLOWER]: Advancing to term 89
I20250111 09:01:46.559834 17613 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 89 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 89 because replica has last-logged OpId of term: 88 index: 1608, which is greater than that of the candidate, which has last-logged OpId of term: 88 index: 1604.
I20250111 09:01:46.561075 17337 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 90 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:46.561344 17340 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 90 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 89. Current term is 90. Ops: [89.1655-89.1656]
I20250111 09:01:46.561434 17337 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 90 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:46.561425 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 89 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:46.561880 18287 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: INVALID_TERM, Last received: 89.1654, Next index: 1655, Last known committed idx: 1654, Time since last communication: 0.000s
I20250111 09:01:46.562197 18262 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 89 LEADER]: Stepping down as leader of term 89
I20250111 09:01:46.562291 18262 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 89 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:46.562346 18292 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 89 FOLLOWER]: Leader election lost for term 89. Reason: could not achieve majority
I20250111 09:01:46.562507 18262 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1654, Committed index: 1654, Last appended: 89.1657, Last appended by leader: 1657, Current term: 89, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:46.562832 18262 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 89 FOLLOWER]: Advancing to term 90
I20250111 09:01:46.563000 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 90 candidate_status { last_received { term: 89 index: 1654 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:46.564190 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 90 candidate_status { last_received { term: 89 index: 1654 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:46.564322 17613 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 90 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 90 because replica has last-logged OpId of term: 89 index: 1657, which is greater than that of the candidate, which has last-logged OpId of term: 89 index: 1654.
I20250111 09:01:46.565833 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 90 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
W20250111 09:01:46.566126 18275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:46.566936 18292 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 90 FOLLOWER]: Leader election lost for term 90. Reason: could not achieve majority
W20250111 09:01:46.568259 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.572100 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.572520 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.574915 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.576347 18294 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 34 similar messages]
W20250111 09:01:46.576408 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.578053 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.582320 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.582935 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.583194 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.588927 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.589893 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.591435 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.597767 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.597949 18295 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.600517 18295 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.608410 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.608649 18296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.611228 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.617437 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.619211 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.624991 18296 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 9 similar messages]
W20250111 09:01:46.625077 18296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.629976 18296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.632449 18296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.639103 18295 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.642449 18295 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.642546 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.652041 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.653931 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.654127 18275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.666062 18296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.667268 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.669306 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.679160 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.681236 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.685784 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.694520 18275 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 86 similar messages]
W20250111 09:01:46.694569 18275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.700096 18275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.702040 18275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.709936 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.720043 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.721174 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.728982 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.740119 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.740180 18295 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.746904 18275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.761832 18275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.762181 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.764809 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.779927 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.783161 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.786834 18295 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.803094 18295 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.803164 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.809721 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.824136 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.826675 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.833662 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.847910 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.847910 18296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.859855 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.871923 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.871990 18295 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.885952 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.898483 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.899456 18275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.910329 18296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.923444 18296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.924417 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.938165 18295 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.952337 18295 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.953362 18295 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:46.968771 18275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.980247 18275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.983784 18275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:46.999430 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.009449 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.012676 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.028332 18295 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.039520 18295 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.042575 18295 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.061056 18275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:47.069525 18275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:47.073807 18275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:47.092484 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.103694 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.105896 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.125818 18295 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.137703 18295 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.137768 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.161211 18275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:47.173838 18275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:47.173919 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:47.196686 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.207805 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.208916 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.231608 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.241905 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.243461 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.268471 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:47.281567 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:47.282756 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:47.306689 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.319597 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.320711 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.348453 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.359520 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.360831 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.390501 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:47.399794 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:47.403010 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:47.429795 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.442024 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.443017 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.473892 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.482949 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.482929 18295 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:47.492393 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:47.492836 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:47.492949 17467 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 88 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:47.493021 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 88 FOLLOWER]: Advancing to term 89
I20250111 09:01:47.493793 17467 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 89 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:47.494077 17467 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 89 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:47.494321 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 89 candidate_status { last_received { term: 88 index: 1605 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:47.494321 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 89 candidate_status { last_received { term: 88 index: 1605 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:47.494460 17613 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 89 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 89 because replica has last-logged OpId of term: 88 index: 1608, which is greater than that of the candidate, which has last-logged OpId of term: 88 index: 1605.
I20250111 09:01:47.494479 17339 raft_consensus.cc:2388] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 89 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 in current term 89: Already voted for candidate fb270e42d7c9400f8c05bb9565db1972 in this term.
I20250111 09:01:47.494555 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:47.494647 17467 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 89 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:47.494697 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 89 FOLLOWER]: Advancing to term 90
I20250111 09:01:47.494722 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 89 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:47.494983 18297 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 89 FOLLOWER]: Leader election lost for term 89. Reason: could not achieve majority
I20250111 09:01:47.495388 17467 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 90 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:47.495671 17467 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 90 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:47.495934 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 90 candidate_status { last_received { term: 89 index: 1656 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:47.495954 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 90 candidate_status { last_received { term: 89 index: 1656 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:47.496075 17339 raft_consensus.cc:2388] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 90 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 in current term 90: Already voted for candidate fb270e42d7c9400f8c05bb9565db1972 in this term.
I20250111 09:01:47.496084 17613 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 90 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 90 because replica has last-logged OpId of term: 89 index: 1657, which is greater than that of the candidate, which has last-logged OpId of term: 89 index: 1656.
I20250111 09:01:47.496419 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 90 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:47.496533 18297 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 90 FOLLOWER]: Leader election lost for term 90. Reason: could not achieve majority
W20250111 09:01:47.518669 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:47.525445 18291 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:47.525665 18275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:47.565920 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.567828 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.572012 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.610277 18294 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 38 similar messages]
W20250111 09:01:47.610337 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.614055 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.617205 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.659732 18275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:47.661571 18275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:47.664688 18275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:47.705746 18293 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 35 similar messages]
W20250111 09:01:47.705811 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.709574 18293 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.709664 18298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:47.714854 18286 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 90 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:01:47.714934 18286 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 90 FOLLOWER]: Advancing to term 91
I20250111 09:01:47.715780 18286 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 91 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:47.716163 18286 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 91 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:47.716498 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 91 candidate_status { last_received { term: 89 index: 1657 } } ignore_live_leader: false dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:47.716537 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 91 candidate_status { last_received { term: 89 index: 1657 } } ignore_live_leader: false dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:47.716835 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 91 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:47.717044 18286 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 91 FOLLOWER]: Leader election lost for term 91. Reason: could not achieve majority
I20250111 09:01:47.722011 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:47.722401 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:47.722481 17613 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 89 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:47.722518 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 89 FOLLOWER]: Advancing to term 90
I20250111 09:01:47.723156 17613 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 90 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:47.723397 17613 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 90 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:47.723649 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 90 candidate_status { last_received { term: 88 index: 1608 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:47.723707 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 90 candidate_status { last_received { term: 88 index: 1608 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:47.723759 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 89 FOLLOWER]: Advancing to term 90
I20250111 09:01:47.723793 17339 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 89 FOLLOWER]: Advancing to term 90
I20250111 09:01:47.724620 17339 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 90 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 90.
I20250111 09:01:47.724620 17467 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 90 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 90.
I20250111 09:01:47.724854 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 90 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:47.724862 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:47.725061 17613 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 91 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:47.725080 18286 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 90 FOLLOWER]: Leader election won for term 90
I20250111 09:01:47.725136 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 91 FOLLOWER]: Advancing to term 92
I20250111 09:01:47.725257 18286 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 90 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:47.725395 18286 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1604, Committed index: 1604, Last appended: 88.1608, Last appended by leader: 1608, Current term: 90, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:47.726012 17613 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 92 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:47.726275 17613 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 92 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:47.726531 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 92 candidate_status { last_received { term: 89 index: 1657 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:47.726647 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 90 FOLLOWER]: Advancing to term 92
I20250111 09:01:47.726663 17339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 92 candidate_status { last_received { term: 89 index: 1657 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:47.726740 17339 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 90 FOLLOWER]: Advancing to term 92
I20250111 09:01:47.726789 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 88 to 90. New cstate: current_term: 90 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:47.727420 17467 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 92 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 92.
I20250111 09:01:47.727530 17339 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 92 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 92.
I20250111 09:01:47.727684 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 92 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:47.727965 18286 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 92 FOLLOWER]: Leader election won for term 92
I20250111 09:01:47.728055 18286 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 92 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:47.728160 18286 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1654, Committed index: 1654, Last appended: 89.1657, Last appended by leader: 1657, Current term: 92, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:47.729612 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 89 to 92. New cstate: current_term: 92 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
W20250111 09:01:47.756471 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.758349 18294 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:47.758431 18295 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:47.806303 17467 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 90 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 88 index: 1605. Preceding OpId from leader: term: 90 index: 1610. (index mismatch)
I20250111 09:01:47.806346 17339 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 90 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 88 index: 1604. Preceding OpId from leader: term: 90 index: 1610. (index mismatch)
I20250111 09:01:47.806682 18299 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1609, Last known committed idx: 1604, Time since last communication: 0.000s
I20250111 09:01:47.806841 18286 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1609, Last known committed idx: 1601, Time since last communication: 0.000s
I20250111 09:01:47.811796 18316 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=2077, attempt_no=0}) took 1260 ms. Trace:
I20250111 09:01:47.811893 18316 rpcz_store.cc:276] 0111 09:01:46.551307 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:46.551343 (+ 36us) service_pool.cc:225] Handling call
0111 09:01:47.811787 (+1260444us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:46.552649 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 2a4e17d0e2894eaa95b650f1af0ce650
0111 09:01:46.552712 (+ 63us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:46.552713 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:01:46.552714 (+ 1us) tablet.cc:659] Decoding operations
0111 09:01:46.552755 (+ 41us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:01:46.552782 (+ 27us) tablet.cc:698] Row locks acquired
0111 09:01:46.552784 (+ 2us) write_op.cc:260] PREPARE: finished
0111 09:01:46.552809 (+ 25us) write_op.cc:270] Start()
0111 09:01:46.552825 (+ 16us) write_op.cc:276] Timestamp: P: 1736586106552765 usec, L: 0
0111 09:01:46.552827 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:01:46.552934 (+ 107us) log.cc:844] Serialized 306 byte log entry
0111 09:01:47.811317 (+1258383us) op_driver.cc:464] REPLICATION: finished
0111 09:01:47.811475 (+ 158us) write_op.cc:301] APPLY: starting
0111 09:01:47.811492 (+ 17us) tablet.cc:1364] starting BulkCheckPresence
0111 09:01:47.811584 (+ 92us) tablet.cc:1367] finished BulkCheckPresence
0111 09:01:47.811585 (+ 1us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:01:47.811618 (+ 33us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:01:47.811651 (+ 33us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=35
0111 09:01:47.811654 (+ 3us) write_op.cc:312] APPLY: finished
0111 09:01:47.811689 (+ 35us) log.cc:844] Serialized 26 byte log entry
0111 09:01:47.811717 (+ 28us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:47.811727 (+ 10us) write_op.cc:454] Released schema lock
0111 09:01:47.811732 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":120,"num_ops":1,"prepare.queue_time_us":1057,"prepare.run_cpu_time_us":332,"prepare.run_wall_time_us":332,"replication_time_us":1258479,"thread_start_us":50,"threads_started":1}]]}
I20250111 09:01:47.814885 18315 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.ParticipateInTransaction from 127.14.61.3:36423 (ReqId={client: c602c61032fa436092728430b0406df9, seq_no=1471, attempt_no=0}) took 1260 ms. Trace:
I20250111 09:01:47.814957 18315 rpcz_store.cc:276] 0111 09:01:46.554180 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:46.554221 (+ 41us) service_pool.cc:225] Handling call
0111 09:01:47.814872 (+1260651us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:46.554382 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:46.554407 (+ 25us) participant_op.cc:199] PREPARE: finished
0111 09:01:46.554426 (+ 19us) participant_op.cc:212] START. Timestamp: 1736586106554380
0111 09:01:46.554428 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:01:46.554493 (+ 65us) log.cc:844] Serialized 67 byte log entry
0111 09:01:47.811510 (+1257017us) op_driver.cc:464] REPLICATION: finished
0111 09:01:47.814748 (+ 3238us) participant_op.cc:265] APPLY: starting
0111 09:01:47.814814 (+ 66us) participant_op.cc:270] APPLY: finished
0111 09:01:47.814837 (+ 23us) log.cc:844] Serialized 15 byte log entry
0111 09:01:47.814853 (+ 16us) participant_op.cc:94] Released txn lock
0111 09:01:47.814860 (+ 7us) participant_op.cc:291] FINISH: Op applied
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3215,"prepare.queue_time_us":34,"prepare.run_cpu_time_us":136,"prepare.run_wall_time_us":136,"replication_time_us":1257070,"thread_start_us":50,"threads_started":1}]]}
I20250111 09:01:47.813477 18316 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=2078, attempt_no=0}) took 1262 ms. Trace:
I20250111 09:01:47.815703 18316 rpcz_store.cc:276] 0111 09:01:46.551490 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:46.551616 (+ 126us) service_pool.cc:225] Handling call
0111 09:01:47.813469 (+1261853us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:46.553346 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 2a4e17d0e2894eaa95b650f1af0ce650
0111 09:01:46.553406 (+ 60us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:46.553410 (+ 4us) write_op.cc:435] Acquired schema lock
0111 09:01:46.553412 (+ 2us) tablet.cc:659] Decoding operations
0111 09:01:46.553455 (+ 43us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:01:46.553486 (+ 31us) tablet.cc:698] Row locks acquired
0111 09:01:46.553489 (+ 3us) write_op.cc:260] PREPARE: finished
0111 09:01:46.553514 (+ 25us) write_op.cc:270] Start()
0111 09:01:46.553532 (+ 18us) write_op.cc:276] Timestamp: P: 1736586106553470 usec, L: 0
0111 09:01:46.553535 (+ 3us) op_driver.cc:348] REPLICATION: starting
0111 09:01:46.553631 (+ 96us) log.cc:844] Serialized 306 byte log entry
0111 09:01:47.811403 (+1257772us) op_driver.cc:464] REPLICATION: finished
0111 09:01:47.812693 (+ 1290us) write_op.cc:301] APPLY: starting
0111 09:01:47.812705 (+ 12us) tablet.cc:1364] starting BulkCheckPresence
0111 09:01:47.812763 (+ 58us) tablet.cc:1367] finished BulkCheckPresence
0111 09:01:47.812764 (+ 1us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:01:47.812790 (+ 26us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:01:47.812815 (+ 25us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=35
0111 09:01:47.812818 (+ 3us) write_op.cc:312] APPLY: finished
0111 09:01:47.812850 (+ 32us) log.cc:844] Serialized 26 byte log entry
0111 09:01:47.813419 (+ 569us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:47.813425 (+ 6us) write_op.cc:454] Released schema lock
0111 09:01:47.813428 (+ 3us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1262,"num_ops":1,"prepare.queue_time_us":1499,"prepare.run_cpu_time_us":320,"prepare.run_wall_time_us":318,"replication_time_us":1257856,"thread_start_us":68,"threads_started":1}]]}
I20250111 09:01:47.817860 18316 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=2065, attempt_no=0}) took 1304 ms. Trace:
I20250111 09:01:47.817912 18316 rpcz_store.cc:276] 0111 09:01:46.513100 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:46.513610 (+ 510us) service_pool.cc:225] Handling call
0111 09:01:47.817852 (+1304242us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:46.545795 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:46.545815 (+ 20us) participant_op.cc:199] PREPARE: finished
0111 09:01:46.545834 (+ 19us) participant_op.cc:212] START. Timestamp: 1736586106545788
0111 09:01:46.545836 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:01:46.545927 (+ 91us) log.cc:844] Serialized 67 byte log entry
0111 09:01:47.811192 (+1265265us) op_driver.cc:464] REPLICATION: finished
0111 09:01:47.811373 (+ 181us) participant_op.cc:265] APPLY: starting
0111 09:01:47.812123 (+ 750us) participant_op.cc:270] APPLY: finished
0111 09:01:47.812141 (+ 18us) log.cc:844] Serialized 15 byte log entry
0111 09:01:47.812156 (+ 15us) participant_op.cc:94] Released txn lock
0111 09:01:47.812161 (+ 5us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:01:47.812301 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 2a4e17d0e2894eaa95b650f1af0ce650
0111 09:01:47.812363 (+ 62us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:47.812365 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:01:47.812366 (+ 1us) tablet.cc:659] Decoding operations
0111 09:01:47.812410 (+ 44us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:01:47.812438 (+ 28us) tablet.cc:698] Row locks acquired
0111 09:01:47.812439 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:01:47.812461 (+ 22us) write_op.cc:270] Start()
0111 09:01:47.812478 (+ 17us) write_op.cc:276] Timestamp: P: 1736586107812442 usec, L: 0
0111 09:01:47.812480 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:01:47.812567 (+ 87us) log.cc:844] Serialized 306 byte log entry
0111 09:01:47.816902 (+ 4335us) op_driver.cc:464] REPLICATION: finished
0111 09:01:47.817601 (+ 699us) write_op.cc:301] APPLY: starting
0111 09:01:47.817613 (+ 12us) tablet.cc:1364] starting BulkCheckPresence
0111 09:01:47.817668 (+ 55us) tablet.cc:1367] finished BulkCheckPresence
0111 09:01:47.817669 (+ 1us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:01:47.817722 (+ 53us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:01:47.817744 (+ 22us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=35
0111 09:01:47.817748 (+ 4us) write_op.cc:312] APPLY: finished
0111 09:01:47.817779 (+ 31us) log.cc:844] Serialized 26 byte log entry
0111 09:01:47.817802 (+ 23us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:47.817808 (+ 6us) write_op.cc:454] Released schema lock
0111 09:01:47.817811 (+ 3us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"thread_start_us":68,"threads_started":1,"txn-participant-registration.queue_time_us":2957,"txn-participant-registration.run_cpu_time_us":340,"txn-participant-registration.run_wall_time_us":29492,"child_traces":[["op",{"apply.queue_time_us":134,"apply.run_cpu_time_us":605,"apply.run_wall_time_us":3348,"prepare.queue_time_us":35,"prepare.run_cpu_time_us":173,"prepare.run_wall_time_us":173,"replication_time_us":1265337,"thread_start_us":82,"threads_started":1,"child_traces":[["op",{"apply.queue_time_us":675,"num_ops":1,"prepare.queue_time_us":28,"prepare.run_cpu_time_us":298,"prepare.run_wall_time_us":297,"replication_time_us":4406}]]}]]}
I20250111 09:01:47.821391 17476 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 92 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 89 index: 1656. Preceding OpId from leader: term: 92 index: 1659. (index mismatch)
I20250111 09:01:47.821722 18299 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1658, Last known committed idx: 1654, Time since last communication: 0.000s
I20250111 09:01:47.824076 17340 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 92 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 89 index: 1654. Preceding OpId from leader: term: 92 index: 1659. (index mismatch)
I20250111 09:01:47.824543 18286 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1658, Last known committed idx: 1654, Time since last communication: 0.000s
I20250111 09:01:47.826483 18315 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=2079, attempt_no=0}) took 1274 ms. Trace:
I20250111 09:01:47.826540 18315 rpcz_store.cc:276] 0111 09:01:46.552296 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:46.552370 (+ 74us) service_pool.cc:225] Handling call
0111 09:01:47.826474 (+1274104us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:46.553022 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 6ab63c6543ad4a0ab83e76267a725edd
0111 09:01:46.553080 (+ 58us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:46.553082 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:01:46.553083 (+ 1us) tablet.cc:659] Decoding operations
0111 09:01:46.553121 (+ 38us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:01:46.553147 (+ 26us) tablet.cc:698] Row locks acquired
0111 09:01:46.553148 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:01:46.553172 (+ 24us) write_op.cc:270] Start()
0111 09:01:46.553188 (+ 16us) write_op.cc:276] Timestamp: P: 1736586106553129 usec, L: 0
0111 09:01:46.553190 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:01:46.553286 (+ 96us) log.cc:844] Serialized 306 byte log entry
0111 09:01:47.826069 (+1272783us) op_driver.cc:464] REPLICATION: finished
0111 09:01:47.826130 (+ 61us) write_op.cc:301] APPLY: starting
0111 09:01:47.826143 (+ 13us) tablet.cc:1364] starting BulkCheckPresence
0111 09:01:47.826229 (+ 86us) tablet.cc:1367] finished BulkCheckPresence
0111 09:01:47.826231 (+ 2us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:01:47.826261 (+ 30us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:01:47.826298 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=35
0111 09:01:47.826306 (+ 8us) write_op.cc:312] APPLY: finished
0111 09:01:47.826362 (+ 56us) log.cc:844] Serialized 26 byte log entry
0111 09:01:47.826408 (+ 46us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:47.826418 (+ 10us) write_op.cc:454] Released schema lock
0111 09:01:47.826423 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":28,"num_ops":1,"prepare.queue_time_us":407,"prepare.run_cpu_time_us":314,"prepare.run_wall_time_us":314,"replication_time_us":1272859}]]}
I20250111 09:01:47.827111 18315 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.ParticipateInTransaction from 127.14.61.3:36423 (ReqId={client: c602c61032fa436092728430b0406df9, seq_no=1472, attempt_no=0}) took 1272 ms. Trace:
I20250111 09:01:47.827162 18315 rpcz_store.cc:276] 0111 09:01:46.554274 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:46.554926 (+ 652us) service_pool.cc:225] Handling call
0111 09:01:47.827100 (+1272174us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:46.555151 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:46.555187 (+ 36us) participant_op.cc:199] PREPARE: finished
0111 09:01:46.555211 (+ 24us) participant_op.cc:212] START. Timestamp: 1736586106555162
0111 09:01:46.555213 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:01:46.555299 (+ 86us) log.cc:844] Serialized 67 byte log entry
0111 09:01:47.826861 (+1271562us) op_driver.cc:464] REPLICATION: finished
0111 09:01:47.826961 (+ 100us) participant_op.cc:265] APPLY: starting
0111 09:01:47.827035 (+ 74us) participant_op.cc:270] APPLY: finished
0111 09:01:47.827061 (+ 26us) log.cc:844] Serialized 15 byte log entry
0111 09:01:47.827077 (+ 16us) participant_op.cc:94] Released txn lock
0111 09:01:47.827085 (+ 8us) participant_op.cc:291] FINISH: Op applied
Metrics: {"child_traces":[["op",{"apply.queue_time_us":70,"prepare.queue_time_us":49,"prepare.run_cpu_time_us":190,"prepare.run_wall_time_us":190,"replication_time_us":1271629}]]}
I20250111 09:01:47.837680 18316 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=2074, attempt_no=0}) took 1294 ms. Trace:
I20250111 09:01:47.837740 18316 rpcz_store.cc:276] 0111 09:01:46.542843 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:46.543727 (+ 884us) service_pool.cc:225] Handling call
0111 09:01:47.837671 (+1293944us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:46.557875 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:46.557893 (+ 18us) participant_op.cc:199] PREPARE: finished
0111 09:01:46.557909 (+ 16us) participant_op.cc:212] START. Timestamp: 1736586106557864
0111 09:01:46.557910 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:01:46.558066 (+ 156us) log.cc:844] Serialized 67 byte log entry
0111 09:01:47.826911 (+1268845us) op_driver.cc:464] REPLICATION: finished
0111 09:01:47.827412 (+ 501us) participant_op.cc:265] APPLY: starting
0111 09:01:47.827953 (+ 541us) participant_op.cc:270] APPLY: finished
0111 09:01:47.827979 (+ 26us) log.cc:844] Serialized 15 byte log entry
0111 09:01:47.828004 (+ 25us) participant_op.cc:94] Released txn lock
0111 09:01:47.828009 (+ 5us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:01:47.828595 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 6ab63c6543ad4a0ab83e76267a725edd
0111 09:01:47.828649 (+ 54us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:47.828650 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:01:47.828651 (+ 1us) tablet.cc:659] Decoding operations
0111 09:01:47.828689 (+ 38us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:01:47.828713 (+ 24us) tablet.cc:698] Row locks acquired
0111 09:01:47.828715 (+ 2us) write_op.cc:260] PREPARE: finished
0111 09:01:47.828736 (+ 21us) write_op.cc:270] Start()
0111 09:01:47.828757 (+ 21us) write_op.cc:276] Timestamp: P: 1736586107828716 usec, L: 0
0111 09:01:47.828759 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:01:47.828853 (+ 94us) log.cc:844] Serialized 306 byte log entry
0111 09:01:47.837240 (+ 8387us) op_driver.cc:464] REPLICATION: finished
0111 09:01:47.837394 (+ 154us) write_op.cc:301] APPLY: starting
0111 09:01:47.837410 (+ 16us) tablet.cc:1364] starting BulkCheckPresence
0111 09:01:47.837470 (+ 60us) tablet.cc:1367] finished BulkCheckPresence
0111 09:01:47.837471 (+ 1us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:01:47.837510 (+ 39us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:01:47.837529 (+ 19us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=35
0111 09:01:47.837532 (+ 3us) write_op.cc:312] APPLY: finished
0111 09:01:47.837563 (+ 31us) log.cc:844] Serialized 26 byte log entry
0111 09:01:47.837601 (+ 38us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:47.837608 (+ 7us) write_op.cc:454] Released schema lock
0111 09:01:47.837612 (+ 4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"txn-participant-registration.queue_time_us":43,"txn-participant-registration.run_cpu_time_us":330,"txn-participant-registration.run_wall_time_us":13903,"child_traces":[["op",{"apply.queue_time_us":471,"apply.run_cpu_time_us":543,"apply.run_wall_time_us":746,"prepare.queue_time_us":52,"prepare.run_cpu_time_us":149,"prepare.run_wall_time_us":222,"replication_time_us":1268984,"child_traces":[["op",{"apply.queue_time_us":119,"num_ops":1,"prepare.queue_time_us":457,"prepare.run_cpu_time_us":303,"prepare.run_wall_time_us":303,"replication_time_us":8462,"spinlock_wait_cycles":24704}]]}]]}
I20250111 09:01:47.840193 18313 mvcc.cc:204] Tried to move back new op lower bound from 7113056697632858112 to 7113056697255530496. Current Snapshot: MvccSnapshot[applied={T|T < 7113056697632858112}]
W20250111 09:01:47.878954 18306 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 32 similar messages]
I20250111 09:01:48.250744 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:48.251770 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:48.251917 17338 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 90 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:48.251981 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 90 FOLLOWER]: Advancing to term 91
I20250111 09:01:48.252812 17338 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 91 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:48.253021 17339 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 91 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 90. Current term is 91. Ops: [90.1734-90.1736]
I20250111 09:01:48.253851 18341 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: INVALID_TERM, Last received: 90.1733, Next index: 1734, Last known committed idx: 1731, Time since last communication: 0.000s
I20250111 09:01:48.254012 18341 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 90 LEADER]: Stepping down as leader of term 90
I20250111 09:01:48.254668 18341 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 90 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:48.254845 18341 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1733, Committed index: 1733, Last appended: 90.1736, Last appended by leader: 1736, Current term: 90, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:48.255079 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.255335 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:48.257042 18341 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 90 FOLLOWER]: Advancing to term 91
I20250111 09:01:48.262928 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 91 candidate_status { last_received { term: 90 index: 1733 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:48.263053 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 90 FOLLOWER]: Advancing to term 91
I20250111 09:01:48.264061 17467 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 91 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 91 because replica has last-logged OpId of term: 90 index: 1736, which is greater than that of the candidate, which has last-logged OpId of term: 90 index: 1733.
I20250111 09:01:48.262634 17338 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 91 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:48.265101 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 91 candidate_status { last_received { term: 90 index: 1733 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:48.265239 17613 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 91 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 91 because replica has last-logged OpId of term: 90 index: 1736, which is greater than that of the candidate, which has last-logged OpId of term: 90 index: 1733.
I20250111 09:01:48.265617 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 91 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:48.266027 17336 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:48.266034 18307 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 91 FOLLOWER]: Leader election lost for term 91. Reason: could not achieve majority
I20250111 09:01:48.266130 17336 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 92 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:48.266191 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 92 FOLLOWER]: Advancing to term 93
I20250111 09:01:48.267545 17336 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 93 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:48.268041 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 93 candidate_status { last_received { term: 92 index: 1780 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:48.268512 17336 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 93 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:48.268903 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 93 candidate_status { last_received { term: 92 index: 1780 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:48.269021 17613 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 92 LEADER]: Stepping down as leader of term 92
I20250111 09:01:48.269058 17613 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 92 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:48.269166 17613 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1780, Committed index: 1780, Last appended: 92.1781, Last appended by leader: 1781, Current term: 92, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:48.269251 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 92 FOLLOWER]: Advancing to term 93
I20250111 09:01:48.269968 17338 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 93 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 92. Current term is 93. Ops: []
I20250111 09:01:48.270131 17613 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 93 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 93 because replica has last-logged OpId of term: 92 index: 1781, which is greater than that of the candidate, which has last-logged OpId of term: 92 index: 1780.
I20250111 09:01:48.270617 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 93 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:48.270826 18307 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 93 FOLLOWER]: Leader election lost for term 93. Reason: could not achieve majority
W20250111 09:01:48.271799 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.272042 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.271907 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.273885 18352 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.274284 18350 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.276890 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.277099 18347 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.278280 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.278533 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.280321 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.281684 18354 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.282112 18355 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.282353 18356 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.283527 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.283149 18354 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.285617 18347 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.285992 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.287958 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.288812 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.288810 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.289484 18347 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.289861 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.290712 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.292378 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.292454 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.294054 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.294932 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.295612 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.295637 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.297806 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.298799 18347 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.299993 18347 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.300016 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.301606 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.302425 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.303640 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.305635 18347 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.305632 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.306859 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.308367 18354 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.308419 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.308449 18356 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.313916 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.314451 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.314707 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.316088 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.317205 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.318280 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.318574 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.318862 18354 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.323096 18354 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.324232 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.325500 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.325541 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.326781 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.329778 18347 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.330224 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.330876 18347 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.334048 18347 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.334354 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.335593 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.336480 18354 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.339334 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.341162 18354 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.341295 18356 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.342216 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.345049 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.345305 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.350190 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.350648 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.350804 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.352283 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.354657 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.356166 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.356570 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.359114 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.362411 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.363215 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.365686 18347 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.366328 18336 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.370010 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.370079 18336 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.372845 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.374225 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.374305 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.376679 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.377918 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.381667 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.384342 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.385622 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.386960 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.388051 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.391479 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.394048 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.394424 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.398173 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.399025 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.400756 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.402305 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.404446 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.408910 18336 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.410291 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.413461 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.415311 18336 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.417397 18336 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.418587 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.419054 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.420270 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.426245 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.427348 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.430819 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.432243 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.434382 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.435873 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.438993 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.439561 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.443229 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.443544 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.449999 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.450107 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.452291 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.454860 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.457517 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.460282 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.463433 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.463788 18336 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.471091 18336 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.471475 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.472360 18336 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.477212 18350 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.477241 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.482173 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.482178 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.482226 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.489645 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.492333 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.493461 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.496227 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.496248 18356 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.501822 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.501905 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.502377 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.509379 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.513716 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.513898 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.517376 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.518682 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.524784 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.524869 17317 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.525390 18336 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.530973 18336 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.536428 18336 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.539073 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.539155 18350 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.541177 18350 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.545771 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.546888 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.550395 18350 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.551656 18350 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.558698 18350 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.565788 18356 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.566022 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.567221 18356 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.568534 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.574180 18356 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.574219 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.574332 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.582249 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.589884 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.592413 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.593317 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.596102 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.597200 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.597620 18336 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.600840 18336 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.609123 18336 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.616868 18350 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.617812 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.620075 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.621904 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.621963 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.625240 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.625327 18350 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.636669 18350 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.644299 18353 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 47 similar messages]
W20250111 09:01:48.644389 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.645503 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.646636 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.649317 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.650614 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.650859 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.652786 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.664052 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.670722 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.673367 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.676530 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.677143 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.677637 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.680262 18336 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.682124 18336 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:48.691306 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:48.691733 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:48.691845 17476 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 91 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:48.691907 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 91 FOLLOWER]: Advancing to term 92
W20250111 09:01:48.692420 18336 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:48.692806 17476 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 92 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:48.693153 17476 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 92 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:48.693361 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 92 candidate_status { last_received { term: 90 index: 1736 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:48.693457 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 91 FOLLOWER]: Advancing to term 92
I20250111 09:01:48.693490 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 92 candidate_status { last_received { term: 90 index: 1736 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:48.693610 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 91 FOLLOWER]: Advancing to term 92
I20250111 09:01:48.693650 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:48.693725 17476 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 92 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:48.693764 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 92 FOLLOWER]: Advancing to term 93
I20250111 09:01:48.694375 17613 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 92 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 92.
I20250111 09:01:48.694432 17338 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 92 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 92.
I20250111 09:01:48.694432 17476 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 93 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:48.694578 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 92 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:48.694707 17476 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 93 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:48.694885 18357 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 92 FOLLOWER]: Leader election won for term 92
I20250111 09:01:48.695048 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 93 candidate_status { last_received { term: 92 index: 1781 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:48.695088 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 93 candidate_status { last_received { term: 92 index: 1781 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:48.695135 18357 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 92 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:48.695158 17338 raft_consensus.cc:2388] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 93 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 in current term 93: Already voted for candidate fb270e42d7c9400f8c05bb9565db1972 in this term.
I20250111 09:01:48.695281 18357 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1733, Committed index: 1733, Last appended: 90.1736, Last appended by leader: 1736, Current term: 92, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:48.695886 17613 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 93 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 93.
I20250111 09:01:48.696127 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 93 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:48.696336 18358 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 93 FOLLOWER]: Leader election won for term 93
I20250111 09:01:48.696424 18358 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 93 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:48.696548 18358 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1780, Committed index: 1780, Last appended: 92.1781, Last appended by leader: 1781, Current term: 93, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:48.696703 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 90 to 92, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 92 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:48.699244 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 92 to 93, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 93 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:48.702029 18350 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.702100 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:48.703879 17613 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 93 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 92 index: 1781. Preceding OpId from leader: term: 93 index: 1783. (index mismatch)
I20250111 09:01:48.703904 17338 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 93 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 92 index: 1780. Preceding OpId from leader: term: 93 index: 1783. (index mismatch)
I20250111 09:01:48.704203 18357 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1782, Last known committed idx: 1780, Time since last communication: 0.000s
I20250111 09:01:48.704311 18358 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1782, Last known committed idx: 1778, Time since last communication: 0.000s
I20250111 09:01:48.705240 17336 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 92 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 90 index: 1733. Preceding OpId from leader: term: 92 index: 1738. (index mismatch)
I20250111 09:01:48.705703 18358 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1737, Last known committed idx: 1731, Time since last communication: 0.000s
I20250111 09:01:48.706730 17613 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 92 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 90 index: 1736. Preceding OpId from leader: term: 92 index: 1738. (index mismatch)
I20250111 09:01:48.707247 18357 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1737, Last known committed idx: 1733, Time since last communication: 0.000s
W20250111 09:01:48.708351 18349 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 52 similar messages]
W20250111 09:01:48.708423 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:48.709422 18325 mvcc.cc:204] Tried to move back new op lower bound from 7113056701248925696 to 7113056701222354944. Current Snapshot: MvccSnapshot[applied={T|T < 7113056701248925696}]
W20250111 09:01:48.710034 18318 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:48.716396 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.720660 18349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.721482 18350 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:48.724371 18350 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:49.234124 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:49.234990 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:49.235200 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 92 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:49.235327 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 92 FOLLOWER]: Advancing to term 93
I20250111 09:01:49.247963 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 93 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:49.249354 17613 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 93 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 92. Current term is 93. Ops: [92.1868-92.1868]
I20250111 09:01:49.250013 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 93 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:49.250478 18362 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: INVALID_TERM, Last received: 92.1867, Next index: 1868, Last known committed idx: 1867, Time since last communication: 0.000s
I20250111 09:01:49.250604 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 93 candidate_status { last_received { term: 92 index: 1867 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:49.250898 17477 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 92 LEADER]: Stepping down as leader of term 92
I20250111 09:01:49.250960 17477 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 92 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:49.251009 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 93 candidate_status { last_received { term: 92 index: 1867 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:49.251077 17477 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1870, Committed index: 1870, Last appended: 92.1873, Last appended by leader: 1873, Current term: 92, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:49.251123 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 92 FOLLOWER]: Advancing to term 93
I20250111 09:01:49.251602 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 92 FOLLOWER]: Advancing to term 93
I20250111 09:01:49.252162 17336 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 93 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 93 because replica has last-logged OpId of term: 92 index: 1872, which is greater than that of the candidate, which has last-logged OpId of term: 92 index: 1867.
I20250111 09:01:49.252586 17477 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 93 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 93 because replica has last-logged OpId of term: 92 index: 1873, which is greater than that of the candidate, which has last-logged OpId of term: 92 index: 1867.
I20250111 09:01:49.252957 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:49.253095 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 93 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:49.253407 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 93 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:49.253543 18371 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 93 FOLLOWER]: Leader election lost for term 93. Reason: could not achieve majority
I20250111 09:01:49.253731 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 93 FOLLOWER]: Advancing to term 94
I20250111 09:01:49.254359 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 94 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:49.254658 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 94 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:49.254916 17613 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 94 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 93. Current term is 94. Ops: [93.1924-93.1925]
I20250111 09:01:49.255131 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 94 candidate_status { last_received { term: 93 index: 1923 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:49.255156 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 94 candidate_status { last_received { term: 93 index: 1923 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:49.255242 17477 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 93 LEADER]: Stepping down as leader of term 93
I20250111 09:01:49.255244 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 93 FOLLOWER]: Advancing to term 94
I20250111 09:01:49.255290 17477 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 93 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:49.255441 17477 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1923, Committed index: 1923, Last appended: 93.1925, Last appended by leader: 1925, Current term: 93, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:49.255518 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 93 FOLLOWER]: Advancing to term 94
I20250111 09:01:49.256045 17336 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 94 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 94.
I20250111 09:01:49.256229 17477 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 94 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 94 because replica has last-logged OpId of term: 93 index: 1925, which is greater than that of the candidate, which has last-logged OpId of term: 93 index: 1923.
I20250111 09:01:49.256347 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 94 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:49.256731 18371 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 94 FOLLOWER]: Leader election won for term 94
I20250111 09:01:49.256807 18371 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 94 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:49.256829 17337 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 94 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 93. Current term is 94. Ops: [93.1919-93.1925]
I20250111 09:01:49.256907 18371 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1923, Committed index: 1923, Last appended: 93.1923, Last appended by leader: 1923, Current term: 94, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:49.258612 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:49.258831 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 93 to 94, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 94 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
W20250111 09:01:49.259927 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.260275 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.261229 18369 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.012s ago.
W20250111 09:01:49.262064 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.265367 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.265921 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:49.267319 17337 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 94 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 93 index: 1918. Preceding OpId from leader: term: 94 index: 1925. (index mismatch)
I20250111 09:01:49.267731 17477 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 94 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 93 index: 1925. Preceding OpId from leader: term: 94 index: 1925. (term mismatch)
I20250111 09:01:49.267781 17477 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 1924
I20250111 09:01:49.267803 17477 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 93.1925
W20250111 09:01:49.267874 17477 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
I20250111 09:01:49.268342 18371 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1924, Last known committed idx: 1913, Time since last communication: 0.000s
I20250111 09:01:49.268651 18345 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1924, Last known committed idx: 1923, Time since last communication: 0.000s
I20250111 09:01:49.268733 18345 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer 262623ac6df745c785ce267ea39bd265 log is divergent from this leader: its last log entry 93.1924 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1923
I20250111 09:01:49.269290 17477 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 1923
I20250111 09:01:49.269335 17477 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 93.1924
W20250111 09:01:49.269408 17477 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
W20250111 09:01:49.269470 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.272434 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.274931 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.282653 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.282866 18381 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 63 similar messages]
W20250111 09:01:49.283504 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.287515 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.287915 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.289193 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.289562 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.289947 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.293720 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.294468 18378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.296445 18378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.297092 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.299036 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.299116 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.299638 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.300890 18377 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.300925 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.301070 18378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.304581 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.305811 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.307646 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.308012 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.309691 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.309783 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.312423 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.313663 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.316154 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.317363 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.317478 18378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.320060 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.320904 18378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.323704 18378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.325296 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.325412 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.328784 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.329147 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.333443 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.333629 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.334798 18378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.336059 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.338608 18378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.341423 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.344973 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.345335 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.345624 18378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.349503 18378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.353046 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.353781 18378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.354147 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.354983 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.358556 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.359714 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.367475 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.368175 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.368453 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.369987 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.373986 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.374887 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.380823 18378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.381889 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.383481 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.384853 18378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.389997 18378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.390180 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.394966 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.397541 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.399057 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.400898 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.407131 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.407205 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.408910 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.414479 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.415820 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.417579 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.421279 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.422538 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.427065 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.430569 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.433913 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.436995 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.438899 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.441128 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.443320 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.448906 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.454317 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.455307 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.455933 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.457319 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.461208 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.468748 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.474483 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.475154 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.476670 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.478154 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.478817 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.487392 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.495795 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.495864 18378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.496946 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.497228 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.497785 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.506696 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.514755 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.516044 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.517458 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.517768 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.518117 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.530095 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.536710 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.537645 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.538244 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.538585 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.541945 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.555645 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.557631 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.558854 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.562359 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.564349 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.566998 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.580968 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.581087 18381 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.581967 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.589799 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.589828 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.591401 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.604303 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.608016 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.608187 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.614892 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.614917 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.619146 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.631717 18378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.631784 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.633766 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.640903 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.642145 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.647539 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.656490 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.659538 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.660682 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.668081 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.669507 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.675477 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.685467 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.685765 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.690320 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.698000 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.698803 18353 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 53 similar messages]
W20250111 09:01:49.698858 18353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.706252 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.712471 18376 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 47 similar messages]
W20250111 09:01:49.712520 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.716701 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.718683 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.727324 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.729676 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.736935 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.743314 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.747591 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.748663 18383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.757500 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.757544 18376 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:49.766402 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:49.774001 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:49.774412 17337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:49.774503 17337 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 93 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:49.774564 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 93 FOLLOWER]: Advancing to term 94
I20250111 09:01:49.775544 17337 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 94 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:49.775909 17337 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 94 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:49.776279 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 94 candidate_status { last_received { term: 92 index: 1872 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:49.776291 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 94 candidate_status { last_received { term: 92 index: 1872 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:49.776382 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 93 FOLLOWER]: Advancing to term 94
I20250111 09:01:49.776417 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 93 FOLLOWER]: Advancing to term 94
W20250111 09:01:49.776511 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:49.777338 17477 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 94 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 94 because replica has last-logged OpId of term: 92 index: 1873, which is greater than that of the candidate, which has last-logged OpId of term: 92 index: 1872.
I20250111 09:01:49.777451 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 94 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 94.
I20250111 09:01:49.777683 17337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:49.777720 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 94 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:01:49.777782 17337 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 94 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:49.777843 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 94 FOLLOWER]: Advancing to term 95
I20250111 09:01:49.777900 18379 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 94 FOLLOWER]: Leader election won for term 94
I20250111 09:01:49.778033 18379 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 94 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:49.778152 18379 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1870, Committed index: 1870, Last appended: 92.1872, Last appended by leader: 1871, Current term: 94, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:49.778553 17337 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 95 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:49.778825 17337 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 95 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:49.779105 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 95 candidate_status { last_received { term: 94 index: 1932 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:49.779105 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 95 candidate_status { last_received { term: 94 index: 1932 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:49.779186 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 94 FOLLOWER]: Advancing to term 95
I20250111 09:01:49.779186 17614 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 94 LEADER]: Stepping down as leader of term 94
I20250111 09:01:49.779218 17614 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 94 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:49.779341 17614 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1932, Committed index: 1932, Last appended: 94.1932, Last appended by leader: 1932, Current term: 94, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:49.779460 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 94 FOLLOWER]: Advancing to term 95
W20250111 09:01:49.779731 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:49.779896 17477 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 95 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 95.
W20250111 09:01:49.779740 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:49.780202 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 95 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 95.
I20250111 09:01:49.780328 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 95 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:49.780454 18379 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 95 FOLLOWER]: Leader election won for term 95
I20250111 09:01:49.780355 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 92 to 94, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 94 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:49.780530 18379 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 95 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:49.780648 18379 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1932, Committed index: 1932, Last appended: 94.1932, Last appended by leader: 1932, Current term: 95, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:49.783298 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 94 to 95, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 95 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:49.790216 17477 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 94 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 92 index: 1873. Preceding OpId from leader: term: 94 index: 1874. (index mismatch)
I20250111 09:01:49.790258 17614 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 94 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 92 index: 1867. Preceding OpId from leader: term: 94 index: 1874. (index mismatch)
I20250111 09:01:49.790529 18384 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1873, Last known committed idx: 1867, Time since last communication: 0.000s
I20250111 09:01:49.790726 18379 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1873, Last known committed idx: 1870, Time since last communication: 0.000s
I20250111 09:01:49.790824 18379 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer 262623ac6df745c785ce267ea39bd265 log is divergent from this leader: its last log entry 92.1873 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1870
I20250111 09:01:49.791496 17477 raft_consensus.cc:1212] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 94 FOLLOWER]: Deduplicated request from leader. Original: 92.1870->[92.1871-94.1874] Dedup: 92.1872->[94.1873-94.1874]
I20250111 09:01:49.791575 17477 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 1872
I20250111 09:01:49.791599 17477 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 92.1873
W20250111 09:01:49.791679 17477 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
W20250111 09:01:49.798259 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:49.802994 17613 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 95 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 94 index: 1932. Preceding OpId from leader: term: 95 index: 1934. (index mismatch)
I20250111 09:01:49.803673 18391 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1933, Last known committed idx: 1932, Time since last communication: 0.000s
I20250111 09:01:49.803773 17476 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 95 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 94 index: 1932. Preceding OpId from leader: term: 95 index: 1934. (index mismatch)
I20250111 09:01:49.804157 18384 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1933, Last known committed idx: 1932, Time since last communication: 0.000s
W20250111 09:01:49.821419 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.821509 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.821730 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:49.822736 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:50.302220 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:50.302668 17474 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:50.302816 17474 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 94 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:50.302897 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 94 FOLLOWER]: Advancing to term 95
I20250111 09:01:50.304157 17474 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 95 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:50.304708 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 95 candidate_status { last_received { term: 94 index: 2000 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:50.304822 17336 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 94 LEADER]: Stepping down as leader of term 94
I20250111 09:01:50.304893 17336 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 94 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:50.305068 17336 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2000, Committed index: 2000, Last appended: 94.2003, Last appended by leader: 2003, Current term: 94, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:50.305177 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 94 FOLLOWER]: Advancing to term 95
W20250111 09:01:50.305272 18327 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.003s ago.
I20250111 09:01:50.306262 17336 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 95 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 95 because replica has last-logged OpId of term: 94 index: 2003, which is greater than that of the candidate, which has last-logged OpId of term: 94 index: 2000.
I20250111 09:01:50.308081 17474 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 95 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:50.308076 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 95 candidate_status { last_received { term: 94 index: 2000 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:50.308357 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 94 FOLLOWER]: Advancing to term 95
I20250111 09:01:50.308948 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:50.309100 17476 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 95 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:50.309199 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 95 FOLLOWER]: Advancing to term 96
I20250111 09:01:50.309335 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 95 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 95.
I20250111 09:01:50.309620 17611 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 95 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 94. Current term is 95. Ops: [94.2000-94.2003]
I20250111 09:01:50.310312 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 95 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:50.310523 18388 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 95 FOLLOWER]: Leader election won for term 95
I20250111 09:01:50.310672 18388 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 95 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:50.310799 18388 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1999, Committed index: 1999, Last appended: 94.2000, Last appended by leader: 2000, Current term: 95, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:50.312752 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 94 to 95, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 95 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:50.313570 17476 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 96 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:50.314487 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 96 candidate_status { last_received { term: 95 index: 2067 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:50.314560 17476 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 96 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:50.314589 17336 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 95 LEADER]: Stepping down as leader of term 95
I20250111 09:01:50.314630 17336 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 95 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:50.314602 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 96 candidate_status { last_received { term: 95 index: 2067 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:50.314714 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 95 FOLLOWER]: Advancing to term 96
I20250111 09:01:50.314770 17336 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2067, Committed index: 2067, Last appended: 95.2071, Last appended by leader: 2071, Current term: 95, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:50.314869 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 95 FOLLOWER]: Advancing to term 96
I20250111 09:01:50.315623 17613 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 96 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 96.
I20250111 09:01:50.315802 17611 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 95 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 94 index: 1999. Preceding OpId from leader: term: 95 index: 2002. (index mismatch)
I20250111 09:01:50.315886 17336 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 96 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 96 because replica has last-logged OpId of term: 95 index: 2071, which is greater than that of the candidate, which has last-logged OpId of term: 95 index: 2067.
I20250111 09:01:50.316082 17474 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 96 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 95. Current term is 96. Ops: [95.2068-95.2071]
I20250111 09:01:50.316203 17611 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 96 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 95. Current term is 96. Ops: [95.2068-95.2071]
I20250111 09:01:50.316538 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 96 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:50.316686 17336 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 95 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 94 index: 2003. Preceding OpId from leader: term: 95 index: 2002. (term mismatch)
I20250111 09:01:50.316737 17336 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 2001
I20250111 09:01:50.316762 17336 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 94.2002
I20250111 09:01:50.316787 18388 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2001, Last known committed idx: 1995, Time since last communication: 0.000s
W20250111 09:01:50.316830 17336 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Aborted: Op aborted by new leader
I20250111 09:01:50.316975 17336 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted WRITE_OP operation due to leader change: 94.2003
W20250111 09:01:50.317095 17336 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Aborted: Op aborted by new leader
I20250111 09:01:50.317708 18388 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2001, Last known committed idx: 2000, Time since last communication: 0.000s
I20250111 09:01:50.317793 18388 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer fb270e42d7c9400f8c05bb9565db1972 log is divergent from this leader: its last log entry 94.2001 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2000
I20250111 09:01:50.316706 18413 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 96 FOLLOWER]: Leader election won for term 96
I20250111 09:01:50.318531 18413 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 96 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:50.318694 18413 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2064, Committed index: 2064, Last appended: 95.2067, Last appended by leader: 2067, Current term: 96, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:50.318994 17336 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 2000
I20250111 09:01:50.319039 17336 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 94.2001
W20250111 09:01:50.319125 17336 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Aborted: Op aborted by new leader
W20250111 09:01:50.320560 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:50.320569 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:50.322990 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:50.323237 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:50.335640 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 95 to 96, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 96 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:50.644953 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:50.645567 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:50.645718 17611 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 95 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:50.645789 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 95 FOLLOWER]: Advancing to term 96
I20250111 09:01:50.646632 17611 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 96 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:50.646937 17611 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 96 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:50.647219 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 96 candidate_status { last_received { term: 95 index: 2009 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:50.647221 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 96 candidate_status { last_received { term: 95 index: 2009 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:50.647343 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 95 FOLLOWER]: Advancing to term 96
I20250111 09:01:50.647348 17474 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 95 LEADER]: Stepping down as leader of term 95
I20250111 09:01:50.647392 17474 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 95 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:50.647540 17474 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2009, Committed index: 2009, Last appended: 95.2009, Last appended by leader: 2009, Current term: 95, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:50.647635 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:50.647712 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 95 FOLLOWER]: Advancing to term 96
I20250111 09:01:50.647729 17611 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 96 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:50.647774 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 96 FOLLOWER]: Advancing to term 97
I20250111 09:01:50.648311 17336 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 96 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 96.
I20250111 09:01:50.648555 17611 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 97 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:50.648625 17474 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 96 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 96.
I20250111 09:01:50.648707 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 96 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:50.648834 17611 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 97 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:50.648897 18341 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 96 FOLLOWER]: Leader election won for term 96
I20250111 09:01:50.648986 18341 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 96 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:50.649189 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 97 candidate_status { last_received { term: 95 index: 2067 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:50.649153 18341 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2009, Committed index: 2009, Last appended: 95.2009, Last appended by leader: 2009, Current term: 96, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:50.649341 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 96 FOLLOWER]: Advancing to term 97
I20250111 09:01:50.650084 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 97 candidate_status { last_received { term: 95 index: 2067 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:50.650190 17336 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 97 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 97 because replica has last-logged OpId of term: 95 index: 2071, which is greater than that of the candidate, which has last-logged OpId of term: 95 index: 2067.
I20250111 09:01:50.650197 17474 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 96 LEADER]: Stepping down as leader of term 96
I20250111 09:01:50.650239 17474 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 96 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:50.650363 17474 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2064, Committed index: 2064, Last appended: 96.2068, Last appended by leader: 2068, Current term: 96, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:50.650473 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 96 FOLLOWER]: Advancing to term 97
I20250111 09:01:50.650938 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 95 to 96, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 96 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:50.651355 17474 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 97 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 97 because replica has last-logged OpId of term: 96 index: 2068, which is greater than that of the candidate, which has last-logged OpId of term: 95 index: 2067.
I20250111 09:01:50.651646 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 97 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:50.651798 18400 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 97 FOLLOWER]: Leader election lost for term 97. Reason: could not achieve majority
I20250111 09:01:51.055238 17474 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 96 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 95 index: 2009. Preceding OpId from leader: term: 96 index: 2010. (index mismatch)
I20250111 09:01:51.055634 18400 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2010, Last known committed idx: 2009, Time since last communication: 0.000s
I20250111 09:01:51.057787 17336 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 96 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 95 index: 2009. Preceding OpId from leader: term: 96 index: 2010. (index mismatch)
I20250111 09:01:51.058214 18400 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2010, Last known committed idx: 2009, Time since last communication: 0.000s
I20250111 09:01:51.589294 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:51.589800 17336 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:51.589929 17336 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 96 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:51.589988 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 96 FOLLOWER]: Advancing to term 97
I20250111 09:01:51.590744 17336 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 97 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:51.591037 17336 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 97 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:51.591519 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 97 candidate_status { last_received { term: 96 index: 2010 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:51.591516 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 97 candidate_status { last_received { term: 96 index: 2010 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:51.591632 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 96 FOLLOWER]: Advancing to term 97
I20250111 09:01:51.591632 17611 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 96 LEADER]: Stepping down as leader of term 96
I20250111 09:01:51.591691 17611 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 96 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:51.591866 17611 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2010, Committed index: 2010, Last appended: 96.2010, Last appended by leader: 2010, Current term: 96, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:51.592023 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 96 FOLLOWER]: Advancing to term 97
I20250111 09:01:51.592649 17474 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 97 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 97.
I20250111 09:01:51.592934 17611 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 97 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 97.
I20250111 09:01:51.592931 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 97 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:51.593389 18426 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 97 FOLLOWER]: Leader election won for term 97
I20250111 09:01:51.593432 17336 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:51.593530 17336 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 97 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:51.593549 18426 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 97 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:51.593586 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 97 FOLLOWER]: Advancing to term 98
I20250111 09:01:51.593672 18426 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2010, Committed index: 2010, Last appended: 96.2010, Last appended by leader: 2010, Current term: 97, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:51.594359 17336 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 98 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:51.594632 17336 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 98 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:51.594933 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 98 candidate_status { last_received { term: 95 index: 2071 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:51.594970 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 98 candidate_status { last_received { term: 95 index: 2071 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:51.595039 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 97 FOLLOWER]: Advancing to term 98
I20250111 09:01:51.595104 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 97 FOLLOWER]: Advancing to term 98
I20250111 09:01:51.595001 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 96 to 97, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 97 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:51.595986 17611 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 98 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 98.
I20250111 09:01:51.596194 17474 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 98 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 98 because replica has last-logged OpId of term: 96 index: 2068, which is greater than that of the candidate, which has last-logged OpId of term: 95 index: 2071.
I20250111 09:01:51.596261 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 98 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:51.596438 18426 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 98 FOLLOWER]: Leader election won for term 98
I20250111 09:01:51.596526 18426 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 98 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:51.596639 18426 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2067, Committed index: 2067, Last appended: 95.2071, Last appended by leader: 2071, Current term: 98, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:51.598773 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 96 to 98, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 98 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:52.002197 17611 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 97 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 96 index: 2010. Preceding OpId from leader: term: 97 index: 2011. (index mismatch)
I20250111 09:01:52.002557 18426 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2011, Last known committed idx: 2010, Time since last communication: 0.000s
I20250111 09:01:52.004652 17474 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 97 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 96 index: 2010. Preceding OpId from leader: term: 97 index: 2011. (index mismatch)
I20250111 09:01:52.005466 18426 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2011, Last known committed idx: 2010, Time since last communication: 0.000s
I20250111 09:01:52.117126 17474 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 98 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 96 index: 2068. Preceding OpId from leader: term: 98 index: 2072. (index mismatch)
I20250111 09:01:52.117491 18426 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2072, Last known committed idx: 2064, Time since last communication: 0.000s
I20250111 09:01:52.117599 18426 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer 262623ac6df745c785ce267ea39bd265 log is divergent from this leader: its last log entry 96.2068 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2064
I20250111 09:01:52.118249 17474 raft_consensus.cc:1212] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 98 FOLLOWER]: Deduplicated request from leader. Original: 95.2064->[95.2065-98.2072] Dedup: 95.2067->[95.2068-98.2072]
I20250111 09:01:52.118343 17474 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 2067
I20250111 09:01:52.118378 17474 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted NO_OP operation due to leader change: 96.2068
I20250111 09:01:52.118405 17474 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 98 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20250111 09:01:52.119220 18369 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: Cannot bump the last assigned timestamp. Tablet is not in leader mode. Last heard from a leader: 1.800s ago.
I20250111 09:01:52.119349 18369 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.ParticipateInTransaction from 127.14.61.3:54455 (ReqId={client: c602c61032fa436092728430b0406df9, seq_no=1779, attempt_no=1}) took 1792 ms. Trace:
I20250111 09:01:52.119628 18369 rpcz_store.cc:276] 0111 09:01:50.326785 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:50.326835 (+ 50us) service_pool.cc:225] Handling call
0111 09:01:52.119339 (+1792504us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:50.327008 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:52.119196 (+1792188us) op_driver.cc:389] HandleFailure(Illegal state: Cannot bump the last assigned timestamp. Tablet is not in leader mode. Last heard from a leader: 1.800s ago.)
0111 09:01:52.119204 (+ 8us) participant_op.cc:94] Released txn lock
0111 09:01:52.119210 (+ 6us) participant_op.cc:287] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"prepare.queue_time_us":41,"spinlock_wait_cycles":18176}]]}
W20250111 09:01:52.120533 18369 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:52.120628 18369 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50258 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=2605, attempt_no=1}) took 1792 ms. Trace:
I20250111 09:01:52.120654 18369 rpcz_store.cc:276] 0111 09:01:50.327711 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:50.327744 (+ 33us) service_pool.cc:225] Handling call
0111 09:01:52.120614 (+1792870us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:52.120385 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:52.120408 (+ 23us) op_driver.cc:389] HandleFailure(Illegal state: Transaction 333 already open)
0111 09:01:52.120413 (+ 5us) participant_op.cc:94] Released txn lock
0111 09:01:52.120417 (+ 4us) participant_op.cc:287] FINISH: Op aborted
Related trace 'op':
0111 09:01:52.120521 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER)
0111 09:01:52.120525 (+ 4us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:52.120527 (+ 2us) write_op.cc:454] Released schema lock
0111 09:01:52.120528 (+ 1us) write_op.cc:335] FINISH: Op aborted
Metrics: {"mutex_wait_us":37,"spinlock_wait_cycles":2176,"thread_start_us":80,"threads_started":1,"txn-participant-registration.queue_time_us":1995,"txn-participant-registration.run_cpu_time_us":360,"txn-participant-registration.run_wall_time_us":923,"child_traces":[["op",{"prepare.queue_time_us":1789635,"child_traces":[["op",{}]]}]]}
W20250111 09:01:52.121166 18369 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:52.121268 18369 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50258 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=2604, attempt_no=1}) took 1792 ms. Trace:
I20250111 09:01:52.121301 18369 rpcz_store.cc:276] 0111 09:01:50.328837 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:50.328878 (+ 41us) service_pool.cc:225] Handling call
0111 09:01:52.121261 (+1792383us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:52.120966 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:52.120982 (+ 16us) op_driver.cc:389] HandleFailure(Illegal state: Transaction 332 already open)
0111 09:01:52.120985 (+ 3us) participant_op.cc:94] Released txn lock
0111 09:01:52.121040 (+ 55us) participant_op.cc:287] FINISH: Op aborted
Related trace 'op':
0111 09:01:52.121155 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER)
0111 09:01:52.121158 (+ 3us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:52.121160 (+ 2us) write_op.cc:454] Released schema lock
0111 09:01:52.121162 (+ 2us) write_op.cc:335] FINISH: Op aborted
Metrics: {"mutex_wait_us":21,"thread_start_us":115,"threads_started":1,"txn-participant-registration.queue_time_us":1753,"txn-participant-registration.run_cpu_time_us":298,"txn-participant-registration.run_wall_time_us":863,"child_traces":[["op",{"prepare.queue_time_us":1789320,"child_traces":[["op",{}]]}]]}
W20250111 09:01:52.121665 18369 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:52.121757 18369 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50258 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=2609, attempt_no=0}) took 1779 ms. Trace:
I20250111 09:01:52.121788 18369 rpcz_store.cc:276] 0111 09:01:50.342515 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:50.342559 (+ 44us) service_pool.cc:225] Handling call
0111 09:01:52.121751 (+1779192us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:52.121557 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:52.121574 (+ 17us) op_driver.cc:389] HandleFailure(Illegal state: Transaction 335 already open)
0111 09:01:52.121577 (+ 3us) participant_op.cc:94] Released txn lock
0111 09:01:52.121579 (+ 2us) participant_op.cc:287] FINISH: Op aborted
Related trace 'op':
0111 09:01:52.121655 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER)
0111 09:01:52.121658 (+ 3us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:52.121660 (+ 2us) write_op.cc:454] Released schema lock
0111 09:01:52.121661 (+ 1us) write_op.cc:335] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":30,"txn-participant-registration.run_cpu_time_us":264,"txn-participant-registration.run_wall_time_us":1028,"child_traces":[["op",{"prepare.queue_time_us":1777815,"child_traces":[["op",{}]]}]]}
W20250111 09:01:52.122069 18369 tablet_replica.cc:1307] Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.003s ago.: cancelling pending write operations [suppressed 14 similar messages]
W20250111 09:01:52.122119 18369 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.003s ago.
I20250111 09:01:52.122192 18369 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50258 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=2608, attempt_no=0}) took 1782 ms. Trace:
I20250111 09:01:52.122220 18369 rpcz_store.cc:276] 0111 09:01:50.339933 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:50.339973 (+ 40us) service_pool.cc:225] Handling call
0111 09:01:52.122186 (+1782213us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:52.122016 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:52.122032 (+ 16us) participant_op.cc:199] PREPARE: finished
0111 09:01:52.122052 (+ 20us) op_driver.cc:389] HandleFailure(Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.003s ago.)
0111 09:01:52.122056 (+ 4us) participant_op.cc:94] Released txn lock
0111 09:01:52.122062 (+ 6us) participant_op.cc:287] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":1232,"txn-participant-registration.run_cpu_time_us":378,"txn-participant-registration.run_wall_time_us":4363,"child_traces":[["op",{"prepare.queue_time_us":1776331}]]}
I20250111 09:01:52.123883 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:52.124501 18442 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39814 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=2603, attempt_no=0}) took 1818 ms. Trace:
I20250111 09:01:52.124728 18440 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39814 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=2602, attempt_no=0}) took 1820 ms. Trace:
I20250111 09:01:52.124598 18442 rpcz_store.cc:276] 0111 09:01:50.306061 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:50.306460 (+ 399us) service_pool.cc:225] Handling call
0111 09:01:52.124488 (+1818028us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:50.307060 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 6ab63c6543ad4a0ab83e76267a725edd
0111 09:01:50.307118 (+ 58us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:50.307120 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:01:50.307121 (+ 1us) tablet.cc:659] Decoding operations
0111 09:01:50.307160 (+ 39us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:01:50.307204 (+ 44us) tablet.cc:698] Row locks acquired
0111 09:01:50.307205 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:01:50.307233 (+ 28us) write_op.cc:270] Start()
0111 09:01:50.307255 (+ 22us) write_op.cc:276] Timestamp: P: 1736586110307186 usec, L: 0
0111 09:01:50.307259 (+ 4us) op_driver.cc:348] REPLICATION: starting
0111 09:01:50.307373 (+ 114us) log.cc:844] Serialized 306 byte log entry
0111 09:01:52.123823 (+1816450us) op_driver.cc:464] REPLICATION: finished
0111 09:01:52.124177 (+ 354us) write_op.cc:301] APPLY: starting
0111 09:01:52.124192 (+ 15us) tablet.cc:1364] starting BulkCheckPresence
0111 09:01:52.124284 (+ 92us) tablet.cc:1367] finished BulkCheckPresence
0111 09:01:52.124285 (+ 1us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:01:52.124315 (+ 30us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:01:52.124341 (+ 26us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=43
0111 09:01:52.124345 (+ 4us) write_op.cc:312] APPLY: finished
0111 09:01:52.124382 (+ 37us) log.cc:844] Serialized 26 byte log entry
0111 09:01:52.124423 (+ 41us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:52.124432 (+ 9us) write_op.cc:454] Released schema lock
0111 09:01:52.124436 (+ 4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":308,"num_ops":1,"prepare.queue_time_us":308,"prepare.run_cpu_time_us":352,"prepare.run_wall_time_us":353,"replication_time_us":1816542,"thread_start_us":72,"threads_started":1}]]}
I20250111 09:01:52.124795 18440 rpcz_store.cc:276] 0111 09:01:50.304613 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:50.305091 (+ 478us) service_pool.cc:225] Handling call
0111 09:01:52.124717 (+1819626us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:50.305521 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 6ab63c6543ad4a0ab83e76267a725edd
0111 09:01:50.305580 (+ 59us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:50.305582 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:01:50.305584 (+ 2us) tablet.cc:659] Decoding operations
0111 09:01:50.305622 (+ 38us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:01:50.305651 (+ 29us) tablet.cc:698] Row locks acquired
0111 09:01:50.305653 (+ 2us) write_op.cc:260] PREPARE: finished
0111 09:01:50.305680 (+ 27us) write_op.cc:270] Start()
0111 09:01:50.305697 (+ 17us) write_op.cc:276] Timestamp: P: 1736586110305633 usec, L: 0
0111 09:01:50.305699 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:01:50.305797 (+ 98us) log.cc:844] Serialized 306 byte log entry
0111 09:01:52.123642 (+1817845us) op_driver.cc:464] REPLICATION: finished
0111 09:01:52.123839 (+ 197us) write_op.cc:301] APPLY: starting
0111 09:01:52.123857 (+ 18us) tablet.cc:1364] starting BulkCheckPresence
0111 09:01:52.123961 (+ 104us) tablet.cc:1367] finished BulkCheckPresence
0111 09:01:52.123963 (+ 2us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:01:52.124003 (+ 40us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:01:52.124044 (+ 41us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=43
0111 09:01:52.124048 (+ 4us) write_op.cc:312] APPLY: finished
0111 09:01:52.124105 (+ 57us) log.cc:844] Serialized 26 byte log entry
0111 09:01:52.124646 (+ 541us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:52.124656 (+ 10us) write_op.cc:454] Released schema lock
0111 09:01:52.124660 (+ 4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":157,"num_ops":1,"prepare.queue_time_us":207,"prepare.run_cpu_time_us":310,"prepare.run_wall_time_us":309,"replication_time_us":1817916,"spinlock_wait_cycles":6016,"thread_start_us":86,"threads_started":1}]]}
I20250111 09:01:52.126147 17611 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 98 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 95 index: 2067. Preceding OpId from leader: term: 98 index: 2072. (index mismatch)
W20250111 09:01:52.127009 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:52.127277 17474 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:52.127283 18426 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2072, Last known committed idx: 2063, Time since last communication: 0.000s
I20250111 09:01:52.127367 17474 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 97 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:52.127416 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 97 FOLLOWER]: Advancing to term 98
I20250111 09:01:52.128346 17474 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 98 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:52.128654 17474 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 98 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
W20250111 09:01:52.131295 18449 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 48 similar messages]
W20250111 09:01:52.131356 18449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:52.131443 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 98 candidate_status { last_received { term: 97 index: 2011 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:52.131541 17336 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 97 LEADER]: Stepping down as leader of term 97
I20250111 09:01:52.131582 17336 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 97 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:52.131644 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 98 candidate_status { last_received { term: 97 index: 2011 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:52.131732 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 97 FOLLOWER]: Advancing to term 98
I20250111 09:01:52.131697 17336 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2011, Committed index: 2011, Last appended: 97.2011, Last appended by leader: 2011, Current term: 97, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:52.131842 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 97 FOLLOWER]: Advancing to term 98
I20250111 09:01:52.132583 17613 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 98 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 98.
I20250111 09:01:52.132768 17336 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 98 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 98.
I20250111 09:01:52.133129 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 98 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:52.133272 18434 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 98 FOLLOWER]: Leader election won for term 98
I20250111 09:01:52.133419 18434 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 98 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:52.133558 18434 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2011, Committed index: 2011, Last appended: 97.2011, Last appended by leader: 2011, Current term: 98, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:52.133594 17474 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:52.134302 17474 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 98 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:52.134366 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 98 FOLLOWER]: Advancing to term 99
I20250111 09:01:52.135315 17474 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 99 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:52.135612 17474 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 99 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:52.136098 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 99 candidate_status { last_received { term: 98 index: 2073 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:52.136206 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 98 FOLLOWER]: Advancing to term 99
W20250111 09:01:52.136426 18449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:52.136906 18443 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39814 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=2599, attempt_no=0}) took 1832 ms. Trace:
I20250111 09:01:52.137166 17611 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 99 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 99 because replica has last-logged OpId of term: 98 index: 2075, which is greater than that of the candidate, which has last-logged OpId of term: 98 index: 2073.
I20250111 09:01:52.136968 18443 rpcz_store.cc:276] 0111 09:01:50.304449 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:50.304489 (+ 40us) service_pool.cc:225] Handling call
0111 09:01:52.136896 (+1832407us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:50.312813 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:50.312832 (+ 19us) participant_op.cc:199] PREPARE: finished
0111 09:01:50.312850 (+ 18us) participant_op.cc:212] START. Timestamp: 1736586110312803
0111 09:01:50.312852 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:01:50.312939 (+ 87us) log.cc:844] Serialized 67 byte log entry
0111 09:01:52.124052 (+1811113us) op_driver.cc:464] REPLICATION: finished
0111 09:01:52.125798 (+ 1746us) participant_op.cc:265] APPLY: starting
0111 09:01:52.126824 (+ 1026us) participant_op.cc:270] APPLY: finished
0111 09:01:52.126846 (+ 22us) log.cc:844] Serialized 15 byte log entry
0111 09:01:52.126871 (+ 25us) participant_op.cc:94] Released txn lock
0111 09:01:52.126876 (+ 5us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:01:52.128809 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 6ab63c6543ad4a0ab83e76267a725edd
0111 09:01:52.128873 (+ 64us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:01:52.128875 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:01:52.128876 (+ 1us) tablet.cc:659] Decoding operations
0111 09:01:52.128915 (+ 39us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:01:52.128944 (+ 29us) tablet.cc:698] Row locks acquired
0111 09:01:52.128945 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:01:52.128968 (+ 23us) write_op.cc:270] Start()
0111 09:01:52.128997 (+ 29us) write_op.cc:276] Timestamp: P: 1736586112128938 usec, L: 0
0111 09:01:52.129014 (+ 17us) op_driver.cc:348] REPLICATION: starting
0111 09:01:52.129105 (+ 91us) log.cc:844] Serialized 306 byte log entry
0111 09:01:52.136480 (+ 7375us) op_driver.cc:464] REPLICATION: finished
0111 09:01:52.136580 (+ 100us) write_op.cc:301] APPLY: starting
0111 09:01:52.136593 (+ 13us) tablet.cc:1364] starting BulkCheckPresence
0111 09:01:52.136653 (+ 60us) tablet.cc:1367] finished BulkCheckPresence
0111 09:01:52.136655 (+ 2us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:01:52.136699 (+ 44us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:01:52.136723 (+ 24us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=43
0111 09:01:52.136728 (+ 5us) write_op.cc:312] APPLY: finished
0111 09:01:52.136777 (+ 49us) log.cc:844] Serialized 26 byte log entry
0111 09:01:52.136820 (+ 43us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:52.136848 (+ 28us) write_op.cc:454] Released schema lock
0111 09:01:52.136853 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"txn-participant-registration.queue_time_us":703,"txn-participant-registration.run_cpu_time_us":319,"txn-participant-registration.run_wall_time_us":6758,"child_traces":[["op",{"apply.queue_time_us":1718,"apply.run_cpu_time_us":751,"apply.run_wall_time_us":1227,"prepare.queue_time_us":732,"prepare.run_cpu_time_us":167,"prepare.run_wall_time_us":166,"replication_time_us":1811189,"thread_start_us":58,"threads_started":1,"child_traces":[["op",{"apply.queue_time_us":73,"num_ops":1,"prepare.queue_time_us":1798,"prepare.run_cpu_time_us":335,"prepare.run_wall_time_us":336,"replication_time_us":7447}]]}]]}
I20250111 09:01:52.138203 17476 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 99 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 98. Current term is 99. Ops: [98.2074-98.2075]
W20250111 09:01:52.139204 18449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:52.139513 17611 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 99 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 98. Current term is 99. Ops: []
I20250111 09:01:52.139860 18426 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: INVALID_TERM, Last received: 98.2075, Next index: 2076, Last known committed idx: 2073, Time since last communication: 0.000s
I20250111 09:01:52.139991 18426 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 98 LEADER]: Stepping down as leader of term 98
I20250111 09:01:52.140045 18426 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 98 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:52.140168 18426 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2075, Committed index: 2075, Last appended: 98.2076, Last appended by leader: 2076, Current term: 98, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:52.140326 18426 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 98 FOLLOWER]: Advancing to term 99
I20250111 09:01:52.140833 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 99 candidate_status { last_received { term: 98 index: 2073 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:52.141304 17611 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 98 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 97 index: 2011. Preceding OpId from leader: term: 98 index: 2013. (index mismatch)
W20250111 09:01:52.141337 18449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:52.141660 17336 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 99 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 99 because replica has last-logged OpId of term: 98 index: 2076, which is greater than that of the candidate, which has last-logged OpId of term: 98 index: 2073.
I20250111 09:01:52.141641 18454 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2012, Last known committed idx: 2011, Time since last communication: 0.000s
I20250111 09:01:52.141956 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 99 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:52.142135 18434 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 99 FOLLOWER]: Leader election lost for term 99. Reason: could not achieve majority
I20250111 09:01:52.142144 17336 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 98 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 97 index: 2011. Preceding OpId from leader: term: 98 index: 2013. (index mismatch)
W20250111 09:01:52.142189 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:52.142503 18434 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2012, Last known committed idx: 2011, Time since last communication: 0.000s
I20250111 09:01:52.142546 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 97 to 98, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 98 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:52.144933 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.144959 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.146373 18441 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:52.146494 18441 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39814 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=2601, attempt_no=0}) took 1841 ms. Trace:
W20250111 09:01:52.146538 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.147569 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.148404 18452 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 41 similar messages]
W20250111 09:01:52.148463 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:52.146539 18441 rpcz_store.cc:276] 0111 09:01:50.304568 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:50.304861 (+ 293us) service_pool.cc:225] Handling call
0111 09:01:52.146485 (+1841624us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:50.311107 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:50.311127 (+ 20us) participant_op.cc:199] PREPARE: finished
0111 09:01:50.311146 (+ 19us) participant_op.cc:212] START. Timestamp: 1736586110311098
0111 09:01:50.311149 (+ 3us) op_driver.cc:348] REPLICATION: starting
0111 09:01:50.311245 (+ 96us) log.cc:844] Serialized 67 byte log entry
0111 09:01:52.123940 (+1812695us) op_driver.cc:464] REPLICATION: finished
0111 09:01:52.124288 (+ 348us) participant_op.cc:265] APPLY: starting
0111 09:01:52.144731 (+ 20443us) participant_op.cc:270] APPLY: finished
0111 09:01:52.144753 (+ 22us) log.cc:844] Serialized 15 byte log entry
0111 09:01:52.144926 (+ 173us) participant_op.cc:94] Released txn lock
0111 09:01:52.144931 (+ 5us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:01:52.146351 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER)
0111 09:01:52.146359 (+ 8us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:01:52.146363 (+ 4us) write_op.cc:454] Released schema lock
0111 09:01:52.146366 (+ 3us) write_op.cc:335] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":745,"txn-participant-registration.run_cpu_time_us":319,"txn-participant-registration.run_wall_time_us":4438,"child_traces":[["op",{"apply.queue_time_us":318,"prepare.queue_time_us":893,"prepare.run_cpu_time_us":174,"prepare.run_wall_time_us":816,"replication_time_us":1812779,"thread_start_us":74,"threads_started":1,"child_traces":[["op",{}]]}]]}
W20250111 09:01:52.152321 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.153477 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.157789 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.158941 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.159390 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.159861 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.160305 18466 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.160640 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.160640 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.164240 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.164672 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.165560 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.165591 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.166427 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.167196 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.167400 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.168241 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.169502 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.170082 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.170818 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.172214 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.172559 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.175271 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.177629 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.178200 18468 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.178212 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.178736 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.179278 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.179278 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.179678 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.180899 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.183337 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.185181 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.186779 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.187373 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.187399 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.189040 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.189133 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.190660 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.190757 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.195386 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.195864 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.196107 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.197118 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.197633 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.200606 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.200709 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.201332 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.201617 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.204211 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.205500 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.205823 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.205921 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.207294 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.213366 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.213899 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.214905 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.215425 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.215975 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.216466 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.218328 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.219241 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.219826 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.224475 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.225780 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.228281 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.228741 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.229663 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.230937 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.231860 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.232446 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.234537 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.238859 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.239817 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.240445 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.240535 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.243973 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.244542 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.245838 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.247224 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.249413 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.253716 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.253778 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.255126 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.255334 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.257644 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.260593 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.261641 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.263561 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.265789 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.266793 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.268263 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.271112 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.271534 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.271584 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.275185 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.278981 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.282793 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.282881 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.282730 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.284122 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.285526 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.288141 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.288730 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.293110 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.297533 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.298043 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.298187 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.299309 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.300648 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.302403 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.304317 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.304580 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.311203 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.312896 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.316535 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.318010 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.318171 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.319327 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.320751 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.320816 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.324818 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.328510 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.329154 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.335566 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.337256 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.337538 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.339417 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.341223 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.342111 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.344607 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.347479 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.350082 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.356879 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.357358 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.360031 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.360031 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.363096 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.363107 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.365437 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.368748 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.369726 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.377820 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.378814 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.379673 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.381428 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.382684 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.386937 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.387030 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.389899 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.391180 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.400655 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.403245 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.404016 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.405475 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.405611 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.410846 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.411064 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.414542 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.414733 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.424508 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.425379 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.428889 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.430583 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.431054 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.435456 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.435962 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.436836 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.439428 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.447217 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.447388 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.452078 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.454190 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.456931 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.459321 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.460511 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.462031 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.466701 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.471832 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.472034 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.475970 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.480629 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.481632 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.484086 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.488512 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.488776 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.491310 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.496821 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.498828 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.503494 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.506182 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.508069 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.508304 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.515556 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.518147 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.518925 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.521602 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.526096 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.530869 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.530918 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.533558 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.538117 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.545230 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.546798 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.547946 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.548139 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.553485 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.559455 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.560133 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.560138 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.566970 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.573516 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.577366 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.577446 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.578621 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.584444 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.587684 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.587733 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.592141 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.598484 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.603957 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.607494 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.609297 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.609884 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.614269 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.616758 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.617965 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.624583 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.629956 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.632843 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.640162 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.641193 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.641633 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.643924 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.646355 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.648552 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.654382 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.659698 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.664043 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.672911 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.674484 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.675889 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.677168 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.677279 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.682281 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.686959 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.691385 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.695084 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.703583 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.709400 18453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.709471 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.710481 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.711055 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.714558 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.720299 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.726773 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.728469 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.739935 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.742038 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.743320 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.745139 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.745661 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.751380 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.755265 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.758507 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.761719 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.772830 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.777710 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.778836 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.781383 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.781972 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.785197 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.792872 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.793195 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.798439 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.810016 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.813555 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.814673 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.817593 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.819463 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.821998 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.828868 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.831594 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.836024 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.844631 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.851081 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.851331 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.855662 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.857198 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.860127 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.867798 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.868769 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.872319 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.883816 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.887931 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.890949 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.893000 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.894379 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.900418 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.907836 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.909937 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.910501 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.925143 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.926771 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.927942 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.932643 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:52.935272 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.937503 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.946911 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.951258 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.951548 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.965209 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.966449 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.967636 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.971691 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.974635 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.978148 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.985167 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.991566 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:52.991865 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.004743 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.008713 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.010673 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.014384 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.014863 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.020063 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.025408 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.032155 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.035872 18451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:53.044994 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:53.045521 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:53.045634 17611 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 98 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:53.045701 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 98 FOLLOWER]: Advancing to term 99
I20250111 09:01:53.046501 17611 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 99 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:53.046829 17611 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 99 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:53.047142 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 99 candidate_status { last_received { term: 98 index: 2016 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:53.047191 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 99 candidate_status { last_received { term: 98 index: 2016 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:53.047281 17476 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 98 LEADER]: Stepping down as leader of term 98
I20250111 09:01:53.047323 17476 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 98 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:53.047336 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 98 FOLLOWER]: Advancing to term 99
I20250111 09:01:53.047475 17476 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2016, Committed index: 2016, Last appended: 98.2016, Last appended by leader: 2016, Current term: 98, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:53.047660 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 98 FOLLOWER]: Advancing to term 99
I20250111 09:01:53.047864 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:53.047956 17611 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 99 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:53.048002 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 99 FOLLOWER]: Advancing to term 100
I20250111 09:01:53.048236 17337 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 99 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 99.
I20250111 09:01:53.048692 17476 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 99 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 99.
I20250111 09:01:53.048745 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 99 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:53.048785 17611 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 100 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:53.048921 18400 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 99 FOLLOWER]: Leader election won for term 99
I20250111 09:01:53.049072 18400 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 99 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:53.049187 18400 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2016, Committed index: 2016, Last appended: 98.2016, Last appended by leader: 2016, Current term: 99, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:53.049501 17611 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 100 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:53.049757 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 100 candidate_status { last_received { term: 98 index: 2075 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:53.049821 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 99 FOLLOWER]: Advancing to term 100
I20250111 09:01:53.050048 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 100 candidate_status { last_received { term: 98 index: 2075 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:53.050124 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 99 FOLLOWER]: Advancing to term 100
I20250111 09:01:53.050565 17476 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 100 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 100.
I20250111 09:01:53.050860 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 100 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:53.051034 18400 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 100 FOLLOWER]: Leader election won for term 100
I20250111 09:01:53.051061 17337 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 100 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 100 because replica has last-logged OpId of term: 98 index: 2076, which is greater than that of the candidate, which has last-logged OpId of term: 98 index: 2075.
I20250111 09:01:53.051119 18400 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 100 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:53.051227 18400 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2073, Committed index: 2073, Last appended: 98.2075, Last appended by leader: 2075, Current term: 100, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:53.052541 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 98 to 99, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 99 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:53.055833 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 98 to 100, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 100 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
W20250111 09:01:53.056620 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.058068 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.061484 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.076313 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:53.445665 17476 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 100 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 98 index: 2073. Preceding OpId from leader: term: 100 index: 2076. (index mismatch)
I20250111 09:01:53.446143 18400 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2076, Last known committed idx: 2072, Time since last communication: 0.000s
I20250111 09:01:53.449381 17337 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 100 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 98 index: 2076. Preceding OpId from leader: term: 100 index: 2076. (term mismatch)
I20250111 09:01:53.449445 17337 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 2075
I20250111 09:01:53.449468 17337 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 98.2076
W20250111 09:01:53.449546 17337 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Aborted: Op aborted by new leader
I20250111 09:01:53.449636 17337 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.ParticipateInTransaction from 127.14.61.3:38369 (ReqId={client: c602c61032fa436092728430b0406df9, seq_no=1779, attempt_no=3}) took 1310 ms. Trace:
I20250111 09:01:53.449681 17337 rpcz_store.cc:276] 0111 09:01:52.139064 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:01:52.139239 (+ 175us) service_pool.cc:225] Handling call
0111 09:01:53.449627 (+1310388us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:01:52.139431 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:01:52.139456 (+ 25us) participant_op.cc:199] PREPARE: finished
0111 09:01:52.139466 (+ 10us) participant_op.cc:212] START. Timestamp: 1736586112139435
0111 09:01:52.139467 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:01:52.139541 (+ 74us) log.cc:844] Serialized 77 byte log entry
0111 09:01:53.449520 (+1309979us) op_driver.cc:464] REPLICATION: finished
0111 09:01:53.449526 (+ 6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0111 09:01:53.449535 (+ 9us) participant_op.cc:94] Released txn lock
0111 09:01:53.449542 (+ 7us) participant_op.cc:287] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"prepare.queue_time_us":25,"prepare.run_cpu_time_us":145,"prepare.run_wall_time_us":146,"replication_time_us":1310025}]]}
I20250111 09:01:53.450505 18470 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2076, Last known committed idx: 2075, Time since last communication: 0.000s
W20250111 09:01:53.453739 18484 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 26 similar messages]
W20250111 09:01:53.459118 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.462694 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.466622 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.468310 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.472755 18467 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 65 similar messages]
W20250111 09:01:53.472810 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:53.475500 17338 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 99 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 98 index: 2016. Preceding OpId from leader: term: 99 index: 2018. (index mismatch)
I20250111 09:01:53.478008 18400 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2017, Last known committed idx: 2016, Time since last communication: 0.000s
I20250111 09:01:53.478436 17474 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 99 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 98 index: 2016. Preceding OpId from leader: term: 99 index: 2018. (index mismatch)
I20250111 09:01:53.478966 18400 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2017, Last known committed idx: 2016, Time since last communication: 0.000s
W20250111 09:01:53.487470 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.488118 18471 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 100 similar messages]
W20250111 09:01:53.492944 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:53.668807 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:53.669272 17337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:53.670658 17337 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 99 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:53.670746 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 99 FOLLOWER]: Advancing to term 100
I20250111 09:01:53.671772 17337 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 100 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:53.672123 17337 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 100 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:53.672441 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 100 candidate_status { last_received { term: 99 index: 2071 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:53.672905 17337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:53.673017 17337 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 100 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:53.673084 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 100 FOLLOWER]: Advancing to term 101
I20250111 09:01:53.674039 17337 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 101 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:53.674376 17337 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 101 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:53.675230 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 100 candidate_status { last_received { term: 99 index: 2071 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:53.675349 17611 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 99 LEADER]: Stepping down as leader of term 99
I20250111 09:01:53.675390 17611 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 99 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:53.675508 17611 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2072, Committed index: 2072, Last appended: 99.2073, Last appended by leader: 2073, Current term: 99, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:53.675608 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 99 FOLLOWER]: Advancing to term 100
I20250111 09:01:53.675668 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 101 candidate_status { last_received { term: 100 index: 2147 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:53.676539 17611 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 100 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 100 because replica has last-logged OpId of term: 99 index: 2073, which is greater than that of the candidate, which has last-logged OpId of term: 99 index: 2071.
I20250111 09:01:53.676926 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 101 candidate_status { last_received { term: 100 index: 2147 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:53.677052 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 100 FOLLOWER]: Advancing to term 101
W20250111 09:01:53.677090 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:53.678092 17474 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 101 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 101.
I20250111 09:01:53.678294 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 101 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters: b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:53.678505 18483 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 101 FOLLOWER]: Leader election won for term 101
I20250111 09:01:53.678699 18483 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 101 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:53.678825 18483 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2144, Committed index: 2144, Last appended: 100.2147, Last appended by leader: 2147, Current term: 101, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:53.679324 17474 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 101 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 100. Current term is 101. Ops: [100.2148-100.2148]
I20250111 09:01:53.679644 18470 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: INVALID_TERM, Last received: 100.2147, Next index: 2148, Last known committed idx: 2147, Time since last communication: 0.000s
I20250111 09:01:53.679790 17336 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 101 LEADER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 100. Current term is 101. Ops: [100.2148-100.2148]
I20250111 09:01:53.680138 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 100 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:53.680526 18481 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: INVALID_TERM, Last received: 100.2147, Next index: 2148, Last known committed idx: 2144, Time since last communication: 0.000s
I20250111 09:01:53.680634 18481 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer fb270e42d7c9400f8c05bb9565db1972 log is divergent from this leader: its last log entry 101.2148 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2144
I20250111 09:01:53.680624 18514 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 100 FOLLOWER]: Leader election lost for term 100. Reason: could not achieve majority
I20250111 09:01:53.681022 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 100 to 101, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 101 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:53.681195 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:53.682912 18496 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 100 LEADER]: Stepping down as leader of term 100
I20250111 09:01:53.682971 18496 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 100 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:53.683086 18496 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2147, Committed index: 2147, Last appended: 100.2151, Last appended by leader: 2151, Current term: 100, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:53.683187 18496 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 100 FOLLOWER]: Advancing to term 101
I20250111 09:01:53.685107 17336 raft_consensus.cc:1212] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 101 LEADER]: Deduplicated request from leader. Original: 100.2144->[100.2145-100.2151] Dedup: 100.2147->[100.2148-100.2151]
I20250111 09:01:53.685191 17336 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 101 LEADER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 100. Current term is 101. Ops: [100.2145-100.2151]
W20250111 09:01:53.685408 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.685459 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.685662 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.685837 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.685475 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.686897 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:53.687601 17467 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 101 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 100. Current term is 101. Ops: [100.2148-100.2149]
W20250111 09:01:53.689057 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.689839 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.691365 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.691980 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.693954 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.696179 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.697223 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.697377 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.699455 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.700584 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.700981 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.701802 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.703850 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.704330 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.704452 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.704720 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.706758 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.708930 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.710289 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.710517 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.710732 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.713066 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.715204 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.717309 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.717638 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.717658 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.718416 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.723937 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.724604 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.725272 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.727494 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.727627 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.729709 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.733242 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.733464 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.734702 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.737452 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.738677 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.740803 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.742559 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.742998 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.746793 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.748484 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.750705 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.751866 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.754304 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.756348 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.759740 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.761085 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.763203 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.763607 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.767030 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.770216 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.771129 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.773402 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.777910 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.778683 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.781410 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.784085 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.785607 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.789249 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.791808 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.795392 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.797469 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.798946 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.801831 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.804545 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.810106 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.812860 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.813445 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.813513 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.817023 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.822075 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.824124 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.828239 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.829789 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.831910 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.834965 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.836602 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.839041 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.845947 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.850433 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.850543 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.853814 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.856277 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.858173 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.863126 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.868772 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.871115 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.872188 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.875988 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.879343 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.883105 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.886730 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.891127 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.894330 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.897655 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.899435 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.901886 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.905547 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.913012 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.916217 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.916612 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.920982 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.922168 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.926995 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.934741 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.937184 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.939810 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.940965 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.946559 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.951961 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.957855 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.958429 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.962015 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.962086 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.968575 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.974305 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.981488 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.983980 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:53.985157 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.985396 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.992774 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:53.999667 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.005236 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.010550 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.011302 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.012370 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.016786 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.025192 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.030097 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.034977 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.036337 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.036391 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.042883 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.053668 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.057878 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.060287 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.062628 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.065172 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.069844 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.078508 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.086774 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.087926 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.088989 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.091130 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.097002 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.105562 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.113400 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.115972 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.116029 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.121168 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:54.123960 17474 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 101 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 100 index: 2147. Preceding OpId from leader: term: 101 index: 2148. (index mismatch)
I20250111 09:01:54.124293 18514 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2148, Last known committed idx: 2147, Time since last communication: 0.000s
W20250111 09:01:54.126123 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:54.127462 17611 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 101 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 100 index: 2151. Preceding OpId from leader: term: 101 index: 2148. (term mismatch)
I20250111 09:01:54.127518 17611 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 2147
I20250111 09:01:54.127544 17611 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 100.2148
W20250111 09:01:54.127619 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:01:54.127749 17611 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 100.2149
W20250111 09:01:54.127804 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:01:54.127977 17611 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 100.2150
W20250111 09:01:54.128052 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:01:54.128175 17611 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 100.2151
W20250111 09:01:54.128217 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Aborted: Op aborted by new leader
I20250111 09:01:54.128834 18514 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2148, Last known committed idx: 2147, Time since last communication: 0.000s
W20250111 09:01:54.137197 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.142280 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.144459 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.149742 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.150645 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.150646 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.153165 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.153674 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.154897 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.154917 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.158217 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.158973 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.161777 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.163612 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.164314 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.167287 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.167295 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.168694 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.170295 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.170572 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.171461 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.172950 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.173472 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.173636 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.174942 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.176914 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.177546 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.178715 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.180632 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.181824 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.181943 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.182634 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.182745 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.183344 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.184401 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.186861 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.191198 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.191620 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.192852 18452 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.192978 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.193192 18522 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.198822 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.201809 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.202191 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.205899 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.206760 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.207129 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.207379 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.209758 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.210969 18522 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.212492 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.214643 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.214812 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.215761 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.217664 18522 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.219615 18522 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.220948 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.220971 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.226682 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.227725 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.227777 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.234088 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.234762 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.235647 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.237752 18522 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.238662 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.240175 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.240707 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.246829 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.247509 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.248313 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.248710 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.249341 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.249804 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.252372 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.254338 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.260794 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.263574 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.263773 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.264240 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.267611 18467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.268680 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.268729 18524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.274422 18524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.277750 18524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.279865 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.280139 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.280153 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.281622 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.282691 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.284097 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.284269 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.295897 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.296041 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.296530 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.297343 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.298729 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.301826 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.302178 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.308954 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.310431 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.312570 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.313277 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.316038 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.316169 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.318154 18524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.319386 18524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.320863 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.329587 18524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.330452 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.331770 18524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.331873 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.336287 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.338934 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.340675 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.346414 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.349175 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.350551 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.351584 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.353928 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.355343 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.357944 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.358542 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.359685 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.369062 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.369175 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.369567 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.373185 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.375777 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.379575 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.381574 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.382800 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.389076 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.389783 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.391067 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.394667 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.394721 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.397758 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.403847 18465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.404675 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.407529 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.410259 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.411635 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.414779 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.420568 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.422380 18513 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.426031 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.427778 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.429036 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.430516 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.431628 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:54.434083 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:54.434469 17474 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:54.434561 17474 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 99 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:54.434619 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 99 FOLLOWER]: Advancing to term 100
I20250111 09:01:54.435398 17474 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 100 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:54.435745 17474 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 100 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
W20250111 09:01:54.435914 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:54.436064 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 100 candidate_status { last_received { term: 99 index: 2073 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:54.436213 17336 raft_consensus.cc:2388] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 100 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 in current term 100: Already voted for candidate fb270e42d7c9400f8c05bb9565db1972 in this term.
I20250111 09:01:54.436184 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 100 candidate_status { last_received { term: 99 index: 2073 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:54.436465 17474 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:54.436568 17474 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 101 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:54.436626 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 101 FOLLOWER]: Advancing to term 102
I20250111 09:01:54.437232 17611 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 100 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 100.
I20250111 09:01:54.437489 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 100 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:54.437491 17474 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 102 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:54.437672 18520 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 100 FOLLOWER]: Leader election won for term 100
I20250111 09:01:54.437752 17474 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 102 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:54.437836 18520 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 100 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:54.437975 18520 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2072, Committed index: 2072, Last appended: 99.2073, Last appended by leader: 2073, Current term: 100, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:54.438094 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 102 candidate_status { last_received { term: 101 index: 2160 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:54.438179 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 101 FOLLOWER]: Advancing to term 102
W20250111 09:01:54.438175 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:54.438879 17611 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 102 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 102.
I20250111 09:01:54.439127 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 102 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:54.439275 18520 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 102 FOLLOWER]: Leader election won for term 102
I20250111 09:01:54.439363 18520 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 102 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:54.439415 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 102 candidate_status { last_received { term: 101 index: 2160 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:54.439455 18520 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2160, Committed index: 2160, Last appended: 101.2160, Last appended by leader: 2160, Current term: 102, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:54.439498 17336 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 101 LEADER]: Stepping down as leader of term 101
I20250111 09:01:54.439528 17336 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 101 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:54.439644 17336 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2160, Committed index: 2160, Last appended: 101.2160, Last appended by leader: 2160, Current term: 101, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:54.439752 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 101 FOLLOWER]: Advancing to term 102
I20250111 09:01:54.440584 17336 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 102 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 102.
I20250111 09:01:54.440670 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 99 to 100, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 100 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:54.440891 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 101 to 102, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 102 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:54.443147 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:54.449704 17611 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 100 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 99 index: 2073. Preceding OpId from leader: term: 100 index: 2075. (index mismatch)
I20250111 09:01:54.450003 18525 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2074, Last known committed idx: 2072, Time since last communication: 0.000s
I20250111 09:01:54.450155 17336 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 100 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 99 index: 2071. Preceding OpId from leader: term: 100 index: 2075. (index mismatch)
I20250111 09:01:54.450516 18525 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2074, Last known committed idx: 2069, Time since last communication: 0.000s
I20250111 09:01:54.453804 18505 mvcc.cc:204] Tried to move back new op lower bound from 7113056724784599040 to 7113056724739239936. Current Snapshot: MvccSnapshot[applied={T|T < 7113056721552822272 or (T in {7113056721556623360,7113056724784599040})}]
W20250111 09:01:54.454406 18487 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:54.487332 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.487332 17298 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.499312 18519 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 70 similar messages]
W20250111 09:01:54.499373 18519 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.499423 18522 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:54.500087 17613 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 102 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 101 index: 2160. Preceding OpId from leader: term: 102 index: 2162. (index mismatch)
I20250111 09:01:54.501576 18529 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2161, Last known committed idx: 2160, Time since last communication: 0.000s
I20250111 09:01:54.509418 17338 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 102 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 101 index: 2160. Preceding OpId from leader: term: 102 index: 2162. (index mismatch)
I20250111 09:01:54.509982 18520 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2161, Last known committed idx: 2160, Time since last communication: 0.000s
W20250111 09:01:54.533900 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:54.535260 18535 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 60 similar messages]
W20250111 09:01:54.559626 18495 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 61 similar messages]
I20250111 09:01:55.010150 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:55.010577 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:55.010715 17613 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 100 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:55.010788 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 100 FOLLOWER]: Advancing to term 101
I20250111 09:01:55.011826 17613 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 101 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:55.012140 17613 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 101 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:55.012636 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:55.012807 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 101 candidate_status { last_received { term: 100 index: 2226 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:55.012915 17467 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 100 LEADER]: Stepping down as leader of term 100
I20250111 09:01:55.012954 17467 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 100 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:55.013064 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 101 candidate_status { last_received { term: 100 index: 2226 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:55.013084 17467 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2225, Committed index: 2225, Last appended: 100.2230, Last appended by leader: 2230, Current term: 100, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:55.013172 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 100 FOLLOWER]: Advancing to term 101
I20250111 09:01:55.014070 17467 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 101 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 101 because replica has last-logged OpId of term: 100 index: 2230, which is greater than that of the candidate, which has last-logged OpId of term: 100 index: 2226.
W20250111 09:01:55.014751 18539 consensus_queue.cc:1159] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b5fb421fa3354606aecbeafe55a0565c" responder_term: 100 status { last_received { term: 100 index: 2226 } last_committed_idx: 2225 last_received_current_leader { term: 100 index: 2226 } }
I20250111 09:01:55.014992 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 102 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:55.015071 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 102 FOLLOWER]: Advancing to term 103
I20250111 09:01:55.015887 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 103 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:55.015973 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 101 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:55.016108 18481 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 101 FOLLOWER]: Leader election lost for term 101. Reason: could not achieve majority
I20250111 09:01:55.016434 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 103 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
W20250111 09:01:55.016697 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:55.016691 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 103 candidate_status { last_received { term: 102 index: 2302 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:55.017392 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 103 candidate_status { last_received { term: 102 index: 2302 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:55.017482 17467 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 102 LEADER]: Stepping down as leader of term 102
I20250111 09:01:55.017529 17467 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 102 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:55.017633 17467 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2300, Committed index: 2300, Last appended: 102.2303, Last appended by leader: 2303, Current term: 102, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:55.017763 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 102 FOLLOWER]: Advancing to term 103
I20250111 09:01:55.018357 17467 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 103 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 103 because replica has last-logged OpId of term: 102 index: 2303, which is greater than that of the candidate, which has last-logged OpId of term: 102 index: 2302.
I20250111 09:01:55.018587 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 103 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:55.018729 18481 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 103 FOLLOWER]: Leader election lost for term 103. Reason: could not achieve majority
W20250111 09:01:55.021106 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.022387 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.024907 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.028805 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.030905 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.034889 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.039680 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:55.039731 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:55.040112 17337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:55.040198 17337 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 100 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:55.040253 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 100 FOLLOWER]: Advancing to term 101
W20250111 09:01:55.040491 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:55.040992 17337 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 101 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:55.041292 17337 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 101 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:55.041567 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 101 candidate_status { last_received { term: 100 index: 2226 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:55.041586 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 101 candidate_status { last_received { term: 100 index: 2226 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:55.041692 17467 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 101 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 101 because replica has last-logged OpId of term: 100 index: 2230, which is greater than that of the candidate, which has last-logged OpId of term: 100 index: 2226.
I20250111 09:01:55.041810 17611 raft_consensus.cc:2388] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 101 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 in current term 101: Already voted for candidate b5fb421fa3354606aecbeafe55a0565c in this term.
I20250111 09:01:55.041808 17337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:55.041985 17337 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 102 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:55.042062 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 102 FOLLOWER]: Advancing to term 103
I20250111 09:01:55.042039 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 101 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:55.042259 18545 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 101 FOLLOWER]: Leader election lost for term 101. Reason: could not achieve majority
I20250111 09:01:55.042757 17337 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 103 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:55.043044 17337 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 103 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:55.043494 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 103 candidate_status { last_received { term: 102 index: 2301 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:55.043483 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 103 candidate_status { last_received { term: 102 index: 2301 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:55.043605 17611 raft_consensus.cc:2388] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 103 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 in current term 103: Already voted for candidate b5fb421fa3354606aecbeafe55a0565c in this term.
I20250111 09:01:55.043613 17467 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 103 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 103 because replica has last-logged OpId of term: 102 index: 2303, which is greater than that of the candidate, which has last-logged OpId of term: 102 index: 2301.
I20250111 09:01:55.043871 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 103 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:55.044003 18545 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 103 FOLLOWER]: Leader election lost for term 103. Reason: could not achieve majority
W20250111 09:01:55.046440 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.046464 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.056131 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.056249 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.064581 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.066520 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.075284 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.079937 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.089185 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.092027 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.104475 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.107167 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.119709 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.121014 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.136415 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.138235 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.153640 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.154827 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.168283 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.174710 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.186730 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.193341 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.208778 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.214339 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.226599 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.237519 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.246701 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:55.250434 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:55.250860 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:55.250960 17467 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 101 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:55.251017 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 101 FOLLOWER]: Advancing to term 102
I20250111 09:01:55.251775 17467 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 102 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:55.252072 17467 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 102 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:55.252318 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 102 candidate_status { last_received { term: 100 index: 2230 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:55.252424 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 101 FOLLOWER]: Advancing to term 102
I20250111 09:01:55.252460 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 102 candidate_status { last_received { term: 100 index: 2230 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:55.252583 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 101 FOLLOWER]: Advancing to term 102
I20250111 09:01:55.253341 17337 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 102 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 102.
I20250111 09:01:55.253486 17611 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 102 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 102.
I20250111 09:01:55.253599 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 102 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:55.253774 18539 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 102 FOLLOWER]: Leader election won for term 102
I20250111 09:01:55.253865 18539 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 102 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:55.253952 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:55.254011 18539 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2225, Committed index: 2225, Last appended: 100.2230, Last appended by leader: 2230, Current term: 102, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:55.254086 17467 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 103 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:55.254146 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 103 FOLLOWER]: Advancing to term 104
I20250111 09:01:55.254899 17467 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 104 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:55.255242 17467 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 104 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:55.255421 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 100 to 102. New cstate: current_term: 102 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:55.255560 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 104 candidate_status { last_received { term: 102 index: 2303 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:55.255650 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 103 FOLLOWER]: Advancing to term 104
I20250111 09:01:55.256248 17611 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 104 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 104.
I20250111 09:01:55.256460 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 104 candidate_status { last_received { term: 102 index: 2303 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:55.256505 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 104 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:55.256557 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 103 FOLLOWER]: Advancing to term 104
I20250111 09:01:55.257432 17337 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 104 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 104.
I20250111 09:01:55.257539 18539 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 104 FOLLOWER]: Leader election won for term 104
I20250111 09:01:55.257644 18539 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 104 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:55.257751 18539 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2300, Committed index: 2300, Last appended: 102.2303, Last appended by leader: 2303, Current term: 104, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:55.260011 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 102 to 104. New cstate: current_term: 104 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:55.262219 17611 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 102 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 100 index: 2226. Preceding OpId from leader: term: 102 index: 2232. (index mismatch)
I20250111 09:01:55.262447 17337 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 102 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 100 index: 2226. Preceding OpId from leader: term: 102 index: 2232. (index mismatch)
I20250111 09:01:55.262504 18539 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2231, Last known committed idx: 2225, Time since last communication: 0.000s
I20250111 09:01:55.262724 18538 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2231, Last known committed idx: 2225, Time since last communication: 0.000s
I20250111 09:01:55.268554 17614 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 104 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 102 index: 2302. Preceding OpId from leader: term: 104 index: 2305. (index mismatch)
I20250111 09:01:55.268910 18539 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2304, Last known committed idx: 2300, Time since last communication: 0.000s
I20250111 09:01:55.269356 17335 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 104 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 102 index: 2301. Preceding OpId from leader: term: 104 index: 2305. (index mismatch)
I20250111 09:01:55.271333 18538 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2304, Last known committed idx: 2300, Time since last communication: 0.000s
I20250111 09:01:55.376677 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:55.377187 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:55.377311 17613 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 102 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:55.377372 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 102 FOLLOWER]: Advancing to term 103
I20250111 09:01:55.378167 17613 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 103 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:55.378479 17613 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 103 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:55.378989 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:55.379097 17613 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 104 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:55.379160 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 104 FOLLOWER]: Advancing to term 105
I20250111 09:01:55.379706 17611 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 103 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 102. Current term is 103. Ops: [102.2256-102.2261]
I20250111 09:01:55.380040 17613 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 105 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:55.380352 17613 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 105 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:55.380378 18550 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: INVALID_TERM, Last received: 102.2255, Next index: 2256, Last known committed idx: 2254, Time since last communication: 0.000s
I20250111 09:01:55.380688 18550 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 102 LEADER]: Stepping down as leader of term 102
I20250111 09:01:55.380772 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 103 candidate_status { last_received { term: 102 index: 2255 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:55.380771 18550 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 102 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:55.381095 18550 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2255, Committed index: 2255, Last appended: 102.2261, Last appended by leader: 2261, Current term: 102, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:55.381278 18550 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 102 FOLLOWER]: Advancing to term 103
I20250111 09:01:55.381410 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 105 candidate_status { last_received { term: 104 index: 2331 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:55.381505 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 105 candidate_status { last_received { term: 104 index: 2331 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:55.381541 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 104 FOLLOWER]: Advancing to term 105
I20250111 09:01:55.381608 17467 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 104 LEADER]: Stepping down as leader of term 104
I20250111 09:01:55.381661 17467 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 104 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:55.381793 17467 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2334, Committed index: 2334, Last appended: 104.2334, Last appended by leader: 2334, Current term: 104, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:55.381880 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 104 FOLLOWER]: Advancing to term 105
I20250111 09:01:55.382457 17336 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 105 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 105 because replica has last-logged OpId of term: 104 index: 2334, which is greater than that of the candidate, which has last-logged OpId of term: 104 index: 2331.
I20250111 09:01:55.382756 17467 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 105 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 105 because replica has last-logged OpId of term: 104 index: 2334, which is greater than that of the candidate, which has last-logged OpId of term: 104 index: 2331.
I20250111 09:01:55.382902 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 103 candidate_status { last_received { term: 102 index: 2255 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:55.382983 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 105 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:55.383226 17614 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 105 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 104. Current term is 105. Ops: [104.2332-104.2332]
I20250111 09:01:55.383350 18481 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 105 FOLLOWER]: Leader election lost for term 105. Reason: could not achieve majority
I20250111 09:01:55.383039 17335 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 102 FOLLOWER]: Advancing to term 103
I20250111 09:01:55.385304 17474 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 103 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 103 because replica has last-logged OpId of term: 102 index: 2261, which is greater than that of the candidate, which has last-logged OpId of term: 102 index: 2255.
I20250111 09:01:55.387140 17335 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 103 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 103 because replica has last-logged OpId of term: 102 index: 2260, which is greater than that of the candidate, which has last-logged OpId of term: 102 index: 2255.
I20250111 09:01:55.388455 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 103 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:55.389034 18481 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 103 FOLLOWER]: Leader election lost for term 103. Reason: could not achieve majority
W20250111 09:01:55.389680 18518 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.390172 18537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.390586 18524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.393664 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.393661 18552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.397601 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.398016 18552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.399062 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.403818 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.407497 18524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.407958 18537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.413264 18537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.414520 18552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.415798 18552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.422222 18552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.422329 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.423699 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.429905 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.430876 18537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.432150 18537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.439842 18552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.441573 18537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.445147 18552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.452369 18552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.453717 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.459412 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.464668 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.467007 18537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.473726 18537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.476125 18537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.478840 18552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.489017 18552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.490453 18552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.491732 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.505874 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.506855 18553 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.508170 18537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.521512 18537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.525365 18552 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 28 similar messages]
W20250111 09:01:55.525437 18552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.526079 18537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.539722 18552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.541940 18552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.542037 18553 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.556741 18553 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.560730 18553 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 29 similar messages]
W20250111 09:01:55.560776 18553 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.563876 18537 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 75 similar messages]
W20250111 09:01:55.563921 18537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.576232 18537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.579695 18537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.583276 18552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.598688 18552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.598685 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.606529 18553 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.619767 18553 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.620790 18544 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:55.627552 18537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:55.635665 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:55.636159 17335 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:55.636268 17335 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 103 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:55.636339 17335 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 103 FOLLOWER]: Advancing to term 104
I20250111 09:01:55.637189 17335 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 104 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:55.637476 17335 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 104 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:55.637771 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 104 candidate_status { last_received { term: 102 index: 2260 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:55.637817 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 104 candidate_status { last_received { term: 102 index: 2260 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:55.637872 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 103 FOLLOWER]: Advancing to term 104
I20250111 09:01:55.637934 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 103 FOLLOWER]: Advancing to term 104
I20250111 09:01:55.638036 17335 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:55.638152 17335 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 105 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:55.638206 17335 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 105 FOLLOWER]: Advancing to term 106
I20250111 09:01:55.638651 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 104 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 104.
I20250111 09:01:55.638921 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 104 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:55.638950 17474 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 104 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 104 because replica has last-logged OpId of term: 102 index: 2261, which is greater than that of the candidate, which has last-logged OpId of term: 102 index: 2260.
I20250111 09:01:55.638971 17335 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 106 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:55.639071 18545 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 104 FOLLOWER]: Leader election won for term 104
I20250111 09:01:55.639209 18545 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 104 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:55.639300 17335 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 106 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:55.639326 18545 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2254, Committed index: 2254, Last appended: 102.2260, Last appended by leader: 2259, Current term: 104, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:55.639744 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 106 candidate_status { last_received { term: 104 index: 2334 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:55.639744 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 106 candidate_status { last_received { term: 104 index: 2334 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:55.639838 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 105 FOLLOWER]: Advancing to term 106
I20250111 09:01:55.639838 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 105 FOLLOWER]: Advancing to term 106
I20250111 09:01:55.640555 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 106 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 106.
I20250111 09:01:55.640553 17474 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 106 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 106.
I20250111 09:01:55.640890 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 102 to 104, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 104 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:55.640987 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 106 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:55.641239 18545 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 106 FOLLOWER]: Leader election won for term 106
I20250111 09:01:55.641331 18545 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 106 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:55.641458 18545 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2334, Committed index: 2334, Last appended: 104.2334, Last appended by leader: 2334, Current term: 106, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:55.643707 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 104 to 106, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 106 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:55.646776 17474 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 104 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 102 index: 2261. Preceding OpId from leader: term: 104 index: 2262. (index mismatch)
I20250111 09:01:55.646888 17614 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 104 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 102 index: 2255. Preceding OpId from leader: term: 104 index: 2262. (index mismatch)
W20250111 09:01:55.646951 18537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:55.647176 18554 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2261, Last known committed idx: 2255, Time since last communication: 0.000s
I20250111 09:01:55.647275 18554 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer 262623ac6df745c785ce267ea39bd265 log is divergent from this leader: its last log entry 102.2261 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2255
I20250111 09:01:55.647418 18545 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2261, Last known committed idx: 2254, Time since last communication: 0.000s
I20250111 09:01:55.647962 17474 raft_consensus.cc:1212] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 104 FOLLOWER]: Deduplicated request from leader. Original: 102.2255->[102.2256-104.2262] Dedup: 102.2260->[104.2261-104.2262]
I20250111 09:01:55.648046 17474 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 2260
I20250111 09:01:55.648075 17474 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 102.2261
W20250111 09:01:55.648177 17474 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
W20250111 09:01:55.665748 18537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:55.676539 17613 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 106 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 104 index: 2331. Preceding OpId from leader: term: 106 index: 2336. (index mismatch)
I20250111 09:01:55.677043 17477 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 106 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 104 index: 2334. Preceding OpId from leader: term: 106 index: 2336. (index mismatch)
I20250111 09:01:55.677578 18554 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2335, Last known committed idx: 2334, Time since last communication: 0.000s
I20250111 09:01:55.677796 18554 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2335, Last known committed idx: 2330, Time since last communication: 0.000s
W20250111 09:01:55.680289 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:55.680373 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:56.151742 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:56.160305 17473 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:56.162948 17473 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 104 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:56.163118 17473 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 104 FOLLOWER]: Advancing to term 105
I20250111 09:01:56.164531 17473 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 105 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:56.165714 17474 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 105 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 104. Current term is 105. Ops: [104.2390-104.2394]
I20250111 09:01:56.165941 17473 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 105 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:56.166345 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 105 candidate_status { last_received { term: 104 index: 2389 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:56.166455 17335 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 104 LEADER]: Stepping down as leader of term 104
I20250111 09:01:56.166494 17335 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 104 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:56.166638 17335 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2390, Committed index: 2390, Last appended: 104.2394, Last appended by leader: 2394, Current term: 104, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:56.166740 17335 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 104 FOLLOWER]: Advancing to term 105
I20250111 09:01:56.166937 17474 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:56.167609 17335 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 105 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 105 because replica has last-logged OpId of term: 104 index: 2394, which is greater than that of the candidate, which has last-logged OpId of term: 104 index: 2389.
I20250111 09:01:56.169112 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 105 candidate_status { last_received { term: 104 index: 2389 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:56.169214 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 104 FOLLOWER]: Advancing to term 105
I20250111 09:01:56.170346 17611 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 105 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 105 because replica has last-logged OpId of term: 104 index: 2392, which is greater than that of the candidate, which has last-logged OpId of term: 104 index: 2389.
I20250111 09:01:56.170672 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 105 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:56.171345 18550 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 105 FOLLOWER]: Leader election lost for term 105. Reason: could not achieve majority
W20250111 09:01:56.172439 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.172955 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:56.173046 17474 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 106 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:56.173122 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 106 FOLLOWER]: Advancing to term 107
I20250111 09:01:56.173982 17474 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 107 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:56.174635 17474 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 107 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:56.174625 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 107 candidate_status { last_received { term: 106 index: 2460 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:56.174717 17337 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 106 LEADER]: Stepping down as leader of term 106
I20250111 09:01:56.174762 17337 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 106 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:56.174901 17337 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2460, Committed index: 2460, Last appended: 106.2461, Last appended by leader: 2461, Current term: 106, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:56.175019 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 106 FOLLOWER]: Advancing to term 107
I20250111 09:01:56.176052 17337 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 107 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 107 because replica has last-logged OpId of term: 106 index: 2461, which is greater than that of the candidate, which has last-logged OpId of term: 106 index: 2460.
W20250111 09:01:56.176548 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.177336 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.177606 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.178952 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.179992 18558 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:56.180334 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 107 candidate_status { last_received { term: 106 index: 2460 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
W20250111 09:01:56.180423 18565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:56.180819 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 106 FOLLOWER]: Advancing to term 107
I20250111 09:01:56.181751 17614 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 107 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 107 because replica has last-logged OpId of term: 106 index: 2461, which is greater than that of the candidate, which has last-logged OpId of term: 106 index: 2460.
I20250111 09:01:56.182368 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 107 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:56.182562 18550 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 107 FOLLOWER]: Leader election lost for term 107. Reason: could not achieve majority
W20250111 09:01:56.183436 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.183593 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.185339 18565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.186408 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.186628 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.187482 18558 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.187994 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.189561 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.189603 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.192840 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.192874 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.192247 18565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.193815 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.194860 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.194962 18565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.197118 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.197883 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.198050 18565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.198213 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.201515 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.202912 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.203665 18558 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.204469 18565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.204625 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.205276 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.206367 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.206375 18558 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.208375 18558 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.211319 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.211591 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.213656 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.213963 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.214378 18565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.215162 18558 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.215011 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.216408 18565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.219993 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.221421 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.222384 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.222514 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.222800 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.223979 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.225567 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.226250 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.230710 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.231544 18558 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.233700 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.234113 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.234117 18558 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.234867 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.235525 18565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.238533 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.243024 18565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.243139 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.243737 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.244318 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.244639 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.247067 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.247961 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.249163 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.253388 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.254622 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.254673 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.256810 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.257616 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.260257 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.261687 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.262475 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.268930 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.268962 18558 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.270277 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.271078 18558 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.273484 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.273653 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.274372 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.276187 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.282859 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.283936 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.284260 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.284315 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.286507 18558 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.287114 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.288676 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.289422 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.298810 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.298811 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.299808 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.300323 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.300366 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.301681 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.305058 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.307739 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.313786 18558 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.315248 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.315897 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.317615 18558 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.318944 18558 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.320339 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.323060 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.325742 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.329562 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.331418 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.331943 18558 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.333402 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.335965 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.338721 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.338975 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.343534 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.346428 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.348776 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.349056 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.352048 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.354643 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.356211 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.357873 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.364228 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.368566 18558 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.368630 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.369053 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.370324 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.372424 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.375512 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.379010 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.384219 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.387635 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.389716 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.389715 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.390892 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.393378 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.397689 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.401274 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.407497 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.407600 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.408699 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.410017 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.412061 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.416724 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.417459 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.420825 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.429383 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.430164 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.431443 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.432651 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.433307 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.437469 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.440222 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.441978 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.452173 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.454281 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.454843 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.454845 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.456259 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.460677 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.463575 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.463654 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.474972 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.477703 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.478098 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.478579 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.480705 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.487553 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.487565 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.487960 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.500666 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.502148 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.502853 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.503093 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.505398 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.513979 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.514276 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.515070 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.525759 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.526726 18543 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 73 similar messages]
W20250111 09:01:56.526777 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.529276 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.529515 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.531529 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.538201 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.542757 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.542871 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.551923 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.552559 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.558084 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.558867 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.558948 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.566043 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.570611 18566 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 42 similar messages]
W20250111 09:01:56.570667 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.570680 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.581483 18568 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 54 similar messages]
W20250111 09:01:56.581537 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.582793 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.587953 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.587949 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.589522 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.594076 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.597430 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.601821 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.612020 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.613739 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.614504 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.616750 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:56.616976 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:56.617446 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:56.617558 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 105 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:56.617616 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 105 FOLLOWER]: Advancing to term 106
I20250111 09:01:56.618549 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 106 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:56.618855 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 106 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:56.619089 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 106 candidate_status { last_received { term: 104 index: 2392 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:56.619125 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 106 candidate_status { last_received { term: 104 index: 2392 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:56.619197 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 105 FOLLOWER]: Advancing to term 106
I20250111 09:01:56.619222 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 105 FOLLOWER]: Advancing to term 106
I20250111 09:01:56.619872 17337 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 106 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 106 because replica has last-logged OpId of term: 104 index: 2394, which is greater than that of the candidate, which has last-logged OpId of term: 104 index: 2392.
I20250111 09:01:56.619947 17467 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 106 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 106.
I20250111 09:01:56.620186 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 106 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:56.620222 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:56.620326 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 107 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:56.620385 18551 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 106 FOLLOWER]: Leader election won for term 106
I20250111 09:01:56.620385 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 107 FOLLOWER]: Advancing to term 108
I20250111 09:01:56.620482 18551 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 106 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
W20250111 09:01:56.620659 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Service unavailable: leader is not yet ready
I20250111 09:01:56.620617 18551 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2390, Committed index: 2390, Last appended: 104.2392, Last appended by leader: 2392, Current term: 106, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:56.620846 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:56.621193 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 108 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:56.621490 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 108 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:56.621759 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 108 candidate_status { last_received { term: 106 index: 2461 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:56.621845 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 107 FOLLOWER]: Advancing to term 108
I20250111 09:01:56.622040 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 108 candidate_status { last_received { term: 106 index: 2461 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:56.622134 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 107 FOLLOWER]: Advancing to term 108
I20250111 09:01:56.622395 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 104 to 106, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 106 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:56.622689 17467 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 108 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 108.
I20250111 09:01:56.622920 17337 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 108 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 108.
I20250111 09:01:56.623003 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 108 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:56.623149 18551 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 108 FOLLOWER]: Leader election won for term 108
I20250111 09:01:56.623246 18551 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 108 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:56.623345 18551 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2460, Committed index: 2460, Last appended: 106.2461, Last appended by leader: 2461, Current term: 108, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:56.625386 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 106 to 108, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 108 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:56.628490 17467 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 106 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 104 index: 2389. Preceding OpId from leader: term: 106 index: 2394. (index mismatch)
I20250111 09:01:56.628520 17337 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 106 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 104 index: 2394. Preceding OpId from leader: term: 106 index: 2394. (term mismatch)
I20250111 09:01:56.628556 17337 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 2393
I20250111 09:01:56.628574 17337 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 104.2394
W20250111 09:01:56.628655 17337 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Aborted: Op aborted by new leader
I20250111 09:01:56.628779 18551 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2393, Last known committed idx: 2388, Time since last communication: 0.000s
I20250111 09:01:56.629174 18551 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2393, Last known committed idx: 2390, Time since last communication: 0.000s
I20250111 09:01:56.629253 18551 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer fb270e42d7c9400f8c05bb9565db1972 log is divergent from this leader: its last log entry 104.2393 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2390
I20250111 09:01:56.629905 17337 raft_consensus.cc:1212] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 106 FOLLOWER]: Deduplicated request from leader. Original: 104.2390->[104.2391-106.2394] Dedup: 104.2392->[106.2393-106.2394]
I20250111 09:01:56.629985 17337 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 2392
I20250111 09:01:56.630012 17337 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted WRITE_OP operation due to leader change: 104.2393
W20250111 09:01:56.630086 17337 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Aborted: Op aborted by new leader
W20250111 09:01:56.634881 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.641975 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.642854 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:56.652307 17467 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 108 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 106 index: 2460. Preceding OpId from leader: term: 108 index: 2464. (index mismatch)
I20250111 09:01:56.653223 18534 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2462, Last known committed idx: 2458, Time since last communication: 0.000s
I20250111 09:01:56.654856 17337 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 108 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 106 index: 2461. Preceding OpId from leader: term: 108 index: 2464. (index mismatch)
I20250111 09:01:56.656222 18534 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2462, Last known committed idx: 2460, Time since last communication: 0.000s
I20250111 09:01:56.950960 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:56.951486 17337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:56.955200 17337 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 106 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:56.955346 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 106 FOLLOWER]: Advancing to term 107
I20250111 09:01:56.956983 17337 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 107 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:56.960295 17335 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 107 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 106. Current term is 107. Ops: [106.2480-106.2482]
I20250111 09:01:56.961264 17337 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 107 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:56.961539 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 107 candidate_status { last_received { term: 106 index: 2479 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:56.961645 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 106 FOLLOWER]: Advancing to term 107
I20250111 09:01:56.962502 17337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:56.962643 17337 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 108 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:56.962766 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 108 FOLLOWER]: Advancing to term 109
I20250111 09:01:56.963176 17477 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 107 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 107 because replica has last-logged OpId of term: 106 index: 2481, which is greater than that of the candidate, which has last-logged OpId of term: 106 index: 2479.
I20250111 09:01:56.963586 17474 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 107 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 106. Current term is 107. Ops: [106.2482-106.2482]
I20250111 09:01:56.963903 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 107 candidate_status { last_received { term: 106 index: 2479 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:56.964022 17614 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 106 LEADER]: Stepping down as leader of term 106
I20250111 09:01:56.964067 17614 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 106 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:56.964176 17614 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2481, Committed index: 2481, Last appended: 106.2483, Last appended by leader: 2483, Current term: 106, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:56.964272 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 106 FOLLOWER]: Advancing to term 107
W20250111 09:01:56.964587 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:56.965065 17337 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 109 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:56.965312 17614 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 107 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 107 because replica has last-logged OpId of term: 106 index: 2483, which is greater than that of the candidate, which has last-logged OpId of term: 106 index: 2479.
I20250111 09:01:56.965492 17335 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 109 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 108. Current term is 109. Ops: [108.2560-108.2560]
I20250111 09:01:56.965842 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 109 candidate_status { last_received { term: 108 index: 2559 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:56.965950 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 108 FOLLOWER]: Advancing to term 109
I20250111 09:01:56.965898 17337 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 109 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:56.966681 17477 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 109 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 109 because replica has last-logged OpId of term: 108 index: 2560, which is greater than that of the candidate, which has last-logged OpId of term: 108 index: 2559.
I20250111 09:01:56.967542 17477 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 109 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 108. Current term is 109. Ops: [108.2561-108.2561]
I20250111 09:01:56.967725 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 109 candidate_status { last_received { term: 108 index: 2559 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:56.967161 18481 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: INVALID_TERM, Last received: 108.2559, Next index: 2560, Last known committed idx: 2558, Time since last communication: 0.000s
W20250111 09:01:56.967849 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:56.967988 17614 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 108 LEADER]: Stepping down as leader of term 108
I20250111 09:01:56.968032 17614 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 108 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:56.968155 17614 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2560, Committed index: 2560, Last appended: 108.2562, Last appended by leader: 2562, Current term: 108, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:56.968250 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 108 FOLLOWER]: Advancing to term 109
I20250111 09:01:56.968292 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 107 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:56.968485 18562 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 107 FOLLOWER]: Leader election lost for term 107. Reason: could not achieve majority
W20250111 09:01:56.968923 18506 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.003s ago.
I20250111 09:01:56.969101 17614 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 109 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 109 because replica has last-logged OpId of term: 108 index: 2562, which is greater than that of the candidate, which has last-logged OpId of term: 108 index: 2559.
W20250111 09:01:56.969244 18577 raft_consensus.cc:931] Couldn't advance consensus term.: Illegal state: Can't advance term to: 109 current term: 109 is higher.
I20250111 09:01:56.970011 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 109 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:56.970194 18562 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 109 FOLLOWER]: Leader election lost for term 109. Reason: could not achieve majority
W20250111 09:01:56.971570 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.971736 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.975193 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.975258 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.976305 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.976341 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.977322 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.980151 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.980238 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.981175 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.981415 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.982281 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.982463 18585 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.984086 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.987612 18586 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.988234 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.989519 18568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.990197 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.990517 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.991087 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.991528 18587 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.991922 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.992709 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.993222 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.996165 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.996177 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:56.996415 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.997169 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.998211 18586 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.998391 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.999321 18586 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:56.999353 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.001206 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.004624 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.005424 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.005440 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.006099 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.007390 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.008527 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.009202 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.009847 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.010141 18586 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.012162 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.014070 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.014207 18586 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.016153 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.018303 18586 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.018524 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.018738 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.018937 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.019405 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.022574 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.022966 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.024292 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.027585 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.027788 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.028962 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.030180 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.030234 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.030485 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.031899 18586 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.034857 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.034961 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.036718 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.037686 18586 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.039939 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.042304 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.042821 18586 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.042851 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.043253 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.046558 18586 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.047297 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.048244 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.050793 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.053790 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.053805 18587 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.054898 18586 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.057415 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.058785 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.060185 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.060936 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.060981 18587 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.062842 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.064865 18587 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.067720 18586 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.068342 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.071808 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.071863 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.072266 18587 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.074884 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.075114 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.077072 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.080168 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.080289 18586 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.080385 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.084971 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.085000 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.088017 18586 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.089452 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.091813 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.093194 18586 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.093601 18585 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.094568 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.095443 18587 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.099881 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.099941 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.102420 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.105953 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.106763 18587 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.108000 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.110471 18587 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.111769 18587 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.113349 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.115722 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.117903 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.120955 18587 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.123131 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.125692 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.126870 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.128582 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.129935 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.131371 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.134979 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.137140 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.140631 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.142693 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.144106 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.145174 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.146696 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.147681 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.151834 18587 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.155177 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.155301 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.160167 18587 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.160692 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.163008 18587 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.163059 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.165109 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.168205 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.170392 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.175787 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.175868 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.178711 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.180734 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.181438 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.182478 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.186615 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.188041 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.190936 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.194995 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.195089 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.200181 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.202304 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.203639 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.203892 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.206588 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.211485 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.212533 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.216008 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.216827 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.221778 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.223875 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.226852 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.227413 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.228803 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.234603 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.235095 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.239614 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.240832 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.243214 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.250298 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.250459 18566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.252367 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.252795 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.257149 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.258695 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.263324 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.266435 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.268682 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.272377 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.272380 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.274896 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.277880 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.281116 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.283144 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.287568 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.292068 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.293289 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.297775 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.298259 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.299736 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.303082 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.304859 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.308336 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.312395 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.317677 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.317790 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.322212 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.325536 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.325713 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.329128 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.329650 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.334728 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.336905 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.342276 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.345162 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.346582 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.352707 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.353039 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.357615 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.357789 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.361078 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.365641 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.367728 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.372043 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.373805 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.379148 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.383560 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.387063 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.388046 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.388190 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.394430 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.395823 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.398221 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.404719 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.409971 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.411341 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.415009 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.415746 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.417225 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.422755 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.425369 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.426476 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.432022 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.439219 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.439488 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.443895 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.444783 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.450222 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.454185 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.458536 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.458757 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.463141 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.469408 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.472993 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.473541 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.475893 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.481382 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.488526 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.491972 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.492062 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.496007 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.500360 18567 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:57.503978 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.504153 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.507316 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.513238 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:57.519471 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:57.520357 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:57.520789 17474 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:57.520900 17474 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 107 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:57.520961 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 107 FOLLOWER]: Advancing to term 108
I20250111 09:01:57.521874 17474 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 108 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:57.522220 17474 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 108 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
W20250111 09:01:57.522320 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:57.522401 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 108 candidate_status { last_received { term: 106 index: 2481 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:57.522495 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 107 FOLLOWER]: Advancing to term 108
I20250111 09:01:57.522580 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 108 candidate_status { last_received { term: 106 index: 2481 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:57.522663 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 107 FOLLOWER]: Advancing to term 108
I20250111 09:01:57.522871 17474 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:57.522953 17474 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 109 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:57.522994 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 109 FOLLOWER]: Advancing to term 110
I20250111 09:01:57.523368 17614 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 108 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 108 because replica has last-logged OpId of term: 106 index: 2483, which is greater than that of the candidate, which has last-logged OpId of term: 106 index: 2481.
I20250111 09:01:57.523434 17337 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 108 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 108.
I20250111 09:01:57.523737 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 108 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters: b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:57.523808 17474 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 110 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:57.524024 18589 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 108 FOLLOWER]: Leader election won for term 108
I20250111 09:01:57.524097 17474 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 110 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:57.524152 18589 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 108 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:57.524281 18589 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2479, Committed index: 2479, Last appended: 106.2481, Last appended by leader: 2481, Current term: 108, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:57.524541 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 110 candidate_status { last_received { term: 108 index: 2560 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:57.524585 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 110 candidate_status { last_received { term: 108 index: 2560 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:57.524624 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 109 FOLLOWER]: Advancing to term 110
I20250111 09:01:57.524669 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 109 FOLLOWER]: Advancing to term 110
I20250111 09:01:57.525539 17614 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 110 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 110 because replica has last-logged OpId of term: 108 index: 2562, which is greater than that of the candidate, which has last-logged OpId of term: 108 index: 2560.
I20250111 09:01:57.525563 17337 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 110 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 110.
I20250111 09:01:57.525861 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 110 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters: b5fb421fa3354606aecbeafe55a0565c
I20250111 09:01:57.526005 17614 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 108 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 106 index: 2483. Preceding OpId from leader: term: 108 index: 2483. (term mismatch)
I20250111 09:01:57.526008 18589 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 110 FOLLOWER]: Leader election won for term 110
I20250111 09:01:57.526064 17614 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 2482
I20250111 09:01:57.526099 17614 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 106.2483
I20250111 09:01:57.526098 18589 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 110 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
W20250111 09:01:57.526199 17614 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:01:57.526254 17337 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 108 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 106 index: 2479. Preceding OpId from leader: term: 108 index: 2483. (index mismatch)
I20250111 09:01:57.526219 18589 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2558, Committed index: 2558, Last appended: 108.2560, Last appended by leader: 2560, Current term: 110, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:57.526871 18593 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2482, Last known committed idx: 2478, Time since last communication: 0.000s
I20250111 09:01:57.527160 18592 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2482, Last known committed idx: 2481, Time since last communication: 0.000s
I20250111 09:01:57.527184 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 106 to 108, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 108 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:57.528051 18543 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 55 similar messages]
W20250111 09:01:57.528100 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:57.528519 18592 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 106.2482 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2481
I20250111 09:01:57.529048 17614 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 2481
I20250111 09:01:57.529096 17614 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 106.2482
W20250111 09:01:57.529167 17614 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Aborted: Op aborted by new leader
W20250111 09:01:57.530913 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:57.532202 18600 mvcc.cc:204] Tried to move back new op lower bound from 7113056737381965824 to 7113056737380593664. Current Snapshot: MvccSnapshot[applied={T|T < 7113056737381965824}]
W20250111 09:01:57.533730 18543 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:57.534062 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 108 to 110, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 110 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:57.537807 17335 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 110 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 108 index: 2559. Preceding OpId from leader: term: 110 index: 2562. (index mismatch)
I20250111 09:01:57.538112 17614 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 110 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 108 index: 2562. Preceding OpId from leader: term: 110 index: 2562. (term mismatch)
I20250111 09:01:57.538170 17614 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 2561
I20250111 09:01:57.538194 17614 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 108.2562
I20250111 09:01:57.538173 18592 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2561, Last known committed idx: 2558, Time since last communication: 0.000s
W20250111 09:01:57.538275 17614 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:01:57.538754 18590 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2561, Last known committed idx: 2560, Time since last communication: 0.000s
I20250111 09:01:57.538851 18590 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 108.2561 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2560
I20250111 09:01:57.539741 17613 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 2560
I20250111 09:01:57.539790 17613 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 108.2561
W20250111 09:01:57.539898 17613 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:01:57.541450 18597 mvcc.cc:204] Tried to move back new op lower bound from 7113056737430446080 to 7113056737389776896. Current Snapshot: MvccSnapshot[applied={T|T < 7113056735020957696}]
W20250111 09:01:57.650110 18609 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 50 similar messages]
W20250111 09:01:57.653096 18614 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 72 similar messages]
I20250111 09:01:58.117697 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:58.118090 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:58.118209 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 108 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:58.118264 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 108 FOLLOWER]: Advancing to term 109
I20250111 09:01:58.119247 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 109 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:58.119563 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 109 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:58.119666 17611 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 109 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 108. Current term is 109. Ops: [108.2633-108.2634]
I20250111 09:01:58.120111 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:58.120213 17611 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 110 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:58.120270 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 110 FOLLOWER]: Advancing to term 111
I20250111 09:01:58.121023 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 109 candidate_status { last_received { term: 108 index: 2632 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:58.121130 17474 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 108 LEADER]: Stepping down as leader of term 108
I20250111 09:01:58.121214 17611 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 111 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:58.121168 17474 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 108 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:58.121515 17611 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 111 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:58.121503 17474 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2636, Committed index: 2636, Last appended: 108.2637, Last appended by leader: 2637, Current term: 108, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:58.121605 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 108 FOLLOWER]: Advancing to term 109
I20250111 09:01:58.122469 17474 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 109 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 109 because replica has last-logged OpId of term: 108 index: 2637, which is greater than that of the candidate, which has last-logged OpId of term: 108 index: 2632.
I20250111 09:01:58.123059 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 109 candidate_status { last_received { term: 108 index: 2632 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
W20250111 09:01:58.123533 18591 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.008s ago.
I20250111 09:01:58.123561 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 111 candidate_status { last_received { term: 110 index: 2709 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:58.123658 17474 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 110 LEADER]: Stepping down as leader of term 110
I20250111 09:01:58.123698 17474 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 110 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:01:58.123786 17474 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2709, Committed index: 2709, Last appended: 110.2710, Last appended by leader: 2710, Current term: 110, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:58.123889 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 110 FOLLOWER]: Advancing to term 111
I20250111 09:01:58.123981 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 111 candidate_status { last_received { term: 110 index: 2709 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
W20250111 09:01:58.124643 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:58.124663 17474 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 111 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 111 because replica has last-logged OpId of term: 110 index: 2710, which is greater than that of the candidate, which has last-logged OpId of term: 110 index: 2709.
I20250111 09:01:58.125411 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 109 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:58.125872 18616 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 109 FOLLOWER]: Leader election lost for term 109. Reason: could not achieve majority
I20250111 09:01:58.126696 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 111 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:58.126996 18624 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 111 FOLLOWER]: Leader election lost for term 111. Reason: could not achieve majority
W20250111 09:01:58.128536 18626 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.128612 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.128885 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.129633 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.130859 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.130937 18630 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.132416 18630 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.133976 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.134819 18630 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.135442 18620 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.136368 18631 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.137086 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.138324 18632 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.139910 18634 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.141218 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.141235 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.141355 18632 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.143304 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.143316 18634 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.144003 18632 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.144110 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.147925 18620 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.147949 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.149459 18631 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.149828 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.149950 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.151062 18620 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.152654 18620 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.154019 18631 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.154048 18630 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.154791 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.156565 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.157840 18630 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.157943 18620 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.158717 18631 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.160775 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.162119 18620 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.162349 18631 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.164268 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.164965 18632 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.167366 18634 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.167702 18632 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.168973 18634 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.169083 18632 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.169489 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.171594 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.172937 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.173365 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.175609 18631 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.178915 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.180066 18620 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.180266 18631 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.182204 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.182451 18620 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.182852 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.185526 18631 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.186136 18620 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.188975 18620 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.190398 18631 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.191730 18631 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.191789 18630 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.193585 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.193790 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.195122 18630 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.196362 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.198776 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.202045 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.204036 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.204078 18634 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.206583 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.208446 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.209228 18634 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.211908 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.212924 18630 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.215701 18620 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.215749 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.216423 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.218621 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.222455 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.224882 18620 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.225445 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.225857 18630 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.228523 18630 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.228543 18631 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.230314 18630 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.232196 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.237579 18630 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.238368 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.241313 18632 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.241313 18634 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.241400 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.245560 18634 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.246719 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.250070 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.252697 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.254734 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.256331 18630 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.258550 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.259941 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.260396 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.263978 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.269062 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.269783 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:58.270171 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
W20250111 09:01:58.270529 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:58.270607 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:58.270706 17340 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 108 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:58.270761 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 108 FOLLOWER]: Advancing to term 109
I20250111 09:01:58.271574 17340 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 109 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:58.271883 17340 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 109 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:58.272150 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 109 candidate_status { last_received { term: 108 index: 2637 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:58.272212 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 109 candidate_status { last_received { term: 108 index: 2637 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:58.272318 17613 raft_consensus.cc:2388] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 109 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 in current term 109: Already voted for candidate b5fb421fa3354606aecbeafe55a0565c in this term.
I20250111 09:01:58.272452 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:58.272552 17340 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 110 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:58.272607 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 110 FOLLOWER]: Advancing to term 111
I20250111 09:01:58.273083 17477 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 109 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 109.
I20250111 09:01:58.273233 17340 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 111 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:58.273386 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 109 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters: b5fb421fa3354606aecbeafe55a0565c
W20250111 09:01:58.273398 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:58.273516 17340 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 111 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:58.273761 18635 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 109 FOLLOWER]: Leader election won for term 109
I20250111 09:01:58.273818 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 111 candidate_status { last_received { term: 110 index: 2710 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:58.273947 18635 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 109 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:58.274089 18635 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2636, Committed index: 2636, Last appended: 108.2637, Last appended by leader: 2637, Current term: 109, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:58.274405 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 111 candidate_status { last_received { term: 110 index: 2710 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:58.274525 17613 raft_consensus.cc:2388] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 111 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 in current term 111: Already voted for candidate b5fb421fa3354606aecbeafe55a0565c in this term.
I20250111 09:01:58.274633 17477 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 111 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 111.
I20250111 09:01:58.274820 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 111 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
W20250111 09:01:58.274899 18630 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:58.274976 18635 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 111 FOLLOWER]: Leader election won for term 111
I20250111 09:01:58.275045 18635 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 111 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:58.275190 18635 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2708, Committed index: 2708, Last appended: 110.2710, Last appended by leader: 2710, Current term: 111, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:58.275787 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 108 to 109, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 109 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:58.278627 17613 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 109 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 108 index: 2632. Preceding OpId from leader: term: 109 index: 2639. (index mismatch)
I20250111 09:01:58.278738 17477 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 109 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 108 index: 2637. Preceding OpId from leader: term: 109 index: 2639. (index mismatch)
I20250111 09:01:58.279179 18636 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2638, Last known committed idx: 2631, Time since last communication: 0.000s
I20250111 09:01:58.279762 18636 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2638, Last known committed idx: 2636, Time since last communication: 0.000s
I20250111 09:01:58.283118 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 110 to 111, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 111 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:58.287976 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:58.290360 17474 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 111 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 110 index: 2710. Preceding OpId from leader: term: 111 index: 2712. (index mismatch)
I20250111 09:01:58.291347 17613 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 111 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 110 index: 2709. Preceding OpId from leader: term: 111 index: 2712. (index mismatch)
I20250111 09:01:58.291518 18635 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2711, Last known committed idx: 2709, Time since last communication: 0.000s
W20250111 09:01:58.294195 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.304703 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:58.305099 18636 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2711, Last known committed idx: 2707, Time since last communication: 0.000s
W20250111 09:01:58.306106 18634 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.555433 18597 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 44 similar messages]
I20250111 09:01:58.569887 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:58.570374 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:58.570587 17467 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 109 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:58.570695 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 109 FOLLOWER]: Advancing to term 110
I20250111 09:01:58.571970 17467 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 110 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:58.573094 17467 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 110 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:58.573314 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 110 candidate_status { last_received { term: 109 index: 2714 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:58.573432 17335 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 109 LEADER]: Stepping down as leader of term 109
I20250111 09:01:58.573486 17335 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 109 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:58.573611 17335 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2714, Committed index: 2714, Last appended: 109.2716, Last appended by leader: 2716, Current term: 109, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:58.573732 17335 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 109 FOLLOWER]: Advancing to term 110
I20250111 09:01:58.573783 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 110 candidate_status { last_received { term: 109 index: 2714 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:58.573881 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 109 FOLLOWER]: Advancing to term 110
I20250111 09:01:58.574302 17477 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 110 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 109. Current term is 110. Ops: [109.2715-109.2716]
I20250111 09:01:58.573979 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:58.574667 17467 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 111 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:58.574703 17335 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 110 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 110 because replica has last-logged OpId of term: 109 index: 2716, which is greater than that of the candidate, which has last-logged OpId of term: 109 index: 2714.
I20250111 09:01:58.574811 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 111 FOLLOWER]: Advancing to term 112
I20250111 09:01:58.575138 17611 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 110 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 110 because replica has last-logged OpId of term: 109 index: 2716, which is greater than that of the candidate, which has last-logged OpId of term: 109 index: 2714.
I20250111 09:01:58.575423 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 110 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:58.575654 18622 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 110 FOLLOWER]: Leader election lost for term 110. Reason: could not achieve majority
I20250111 09:01:58.579594 17467 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 112 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:58.580061 17467 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 112 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
W20250111 09:01:58.580049 18641 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:58.580195 17477 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 112 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 111. Current term is 112. Ops: [111.2787-111.2789]
I20250111 09:01:58.580770 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 112 candidate_status { last_received { term: 111 index: 2786 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:58.580816 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 112 candidate_status { last_received { term: 111 index: 2786 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:58.580870 17335 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 111 LEADER]: Stepping down as leader of term 111
I20250111 09:01:58.580905 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 111 FOLLOWER]: Advancing to term 112
I20250111 09:01:58.580920 17335 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 111 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:58.581058 17335 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2786, Committed index: 2786, Last appended: 111.2790, Last appended by leader: 2790, Current term: 111, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:58.581163 17335 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 111 FOLLOWER]: Advancing to term 112
I20250111 09:01:58.581847 17611 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 112 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 112 because replica has last-logged OpId of term: 111 index: 2789, which is greater than that of the candidate, which has last-logged OpId of term: 111 index: 2786.
I20250111 09:01:58.582000 17335 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 112 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 112 because replica has last-logged OpId of term: 111 index: 2790, which is greater than that of the candidate, which has last-logged OpId of term: 111 index: 2786.
I20250111 09:01:58.582307 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 112 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:58.582512 18622 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 112 FOLLOWER]: Leader election lost for term 112. Reason: could not achieve majority
W20250111 09:01:58.582897 17320 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.582908 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.585731 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.588150 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.588444 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.592337 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.593412 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.594533 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.595727 18634 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.597489 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.599624 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.602285 18641 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.603060 18642 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.603840 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.608773 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.610443 18642 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.611950 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.612839 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.618297 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.618587 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.620291 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.621800 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.628362 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.630563 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.632761 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.633973 18642 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.639000 18642 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.640745 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.640754 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.642566 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.647882 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.649678 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.651484 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.656894 18633 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 32 similar messages]
W20250111 09:01:58.656970 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.661716 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.662117 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.664860 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.671010 18642 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.674718 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.676973 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.677570 18642 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.687371 18569 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 80 similar messages]
W20250111 09:01:58.687425 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.687805 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.689339 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.693486 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.701910 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.704583 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.705235 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.709690 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.720901 18642 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.721455 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.722193 18642 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.725658 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.736959 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.738095 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.739353 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.744035 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.753044 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.758569 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.760546 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.762678 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.773926 18642 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.780205 18642 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.781723 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.783794 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.792239 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.799129 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.803571 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.806358 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.811759 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.821516 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:58.827620 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.829674 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.835098 18642 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.843397 18642 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.850481 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.852521 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.856117 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:58.864874 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:01:58.865391 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:58.865489 17611 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 110 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:58.865557 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 110 FOLLOWER]: Advancing to term 111
I20250111 09:01:58.866498 17611 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 111 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:58.866886 17611 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 111 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:58.867133 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 111 candidate_status { last_received { term: 109 index: 2716 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:58.867241 17335 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 110 FOLLOWER]: Advancing to term 111
I20250111 09:01:58.867280 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 111 candidate_status { last_received { term: 109 index: 2716 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:58.867379 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 110 FOLLOWER]: Advancing to term 111
I20250111 09:01:58.868208 17335 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 111 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 111.
I20250111 09:01:58.868223 17467 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 111 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 111.
I20250111 09:01:58.868449 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:58.868458 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 111 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:58.868558 17611 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 112 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:58.868609 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 112 FOLLOWER]: Advancing to term 113
I20250111 09:01:58.868649 18624 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 111 FOLLOWER]: Leader election won for term 111
W20250111 09:01:58.868645 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:58.868726 18624 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 111 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:58.868841 18624 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2714, Committed index: 2714, Last appended: 109.2716, Last appended by leader: 2716, Current term: 111, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:58.869323 17611 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 113 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:58.869582 17611 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 113 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:58.869858 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 113 candidate_status { last_received { term: 111 index: 2789 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:58.869859 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 113 candidate_status { last_received { term: 111 index: 2789 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:58.869951 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 112 FOLLOWER]: Advancing to term 113
I20250111 09:01:58.869951 17335 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 112 FOLLOWER]: Advancing to term 113
I20250111 09:01:58.870025 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 109 to 111, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 111 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:58.870675 17467 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 113 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 113.
I20250111 09:01:58.870719 17335 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 113 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 113 because replica has last-logged OpId of term: 111 index: 2790, which is greater than that of the candidate, which has last-logged OpId of term: 111 index: 2789.
I20250111 09:01:58.870901 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 113 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:58.871050 18624 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 113 FOLLOWER]: Leader election won for term 113
I20250111 09:01:58.871146 18624 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 113 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:58.871243 18624 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2786, Committed index: 2786, Last appended: 111.2789, Last appended by leader: 2789, Current term: 113, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:58.873075 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 111 to 113, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 113 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:58.873939 17335 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 113 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 111 index: 2790. Preceding OpId from leader: term: 113 index: 2791. (index mismatch)
I20250111 09:01:58.874302 18624 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2790, Last known committed idx: 2786, Time since last communication: 0.000s
I20250111 09:01:58.874367 17467 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 113 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 111 index: 2786. Preceding OpId from leader: term: 113 index: 2791. (index mismatch)
I20250111 09:01:58.874385 18624 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer fb270e42d7c9400f8c05bb9565db1972 log is divergent from this leader: its last log entry 111.2790 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2786
I20250111 09:01:58.874922 18624 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2790, Last known committed idx: 2784, Time since last communication: 0.000s
I20250111 09:01:58.874948 17335 raft_consensus.cc:1212] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 113 FOLLOWER]: Deduplicated request from leader. Original: 111.2786->[111.2787-113.2791] Dedup: 111.2789->[113.2790-113.2791]
I20250111 09:01:58.875007 17335 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 2789
I20250111 09:01:58.875038 17335 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted WRITE_OP operation due to leader change: 111.2790
W20250111 09:01:58.875129 17335 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Aborted: Op aborted by new leader
I20250111 09:01:58.880110 17473 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 111 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 109 index: 2714. Preceding OpId from leader: term: 111 index: 2718. (index mismatch)
I20250111 09:01:58.880399 18617 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2717, Last known committed idx: 2714, Time since last communication: 0.000s
W20250111 09:01:58.880779 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:58.881244 17335 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 111 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 109 index: 2716. Preceding OpId from leader: term: 111 index: 2718. (index mismatch)
I20250111 09:01:58.881927 18617 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2717, Last known committed idx: 2714, Time since last communication: 0.000s
W20250111 09:01:58.883080 18649 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:58.884343 18569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:59.144003 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:01:59.144893 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:59.145776 17340 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 111 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:59.145862 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 111 FOLLOWER]: Advancing to term 112
I20250111 09:01:59.146884 17340 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 112 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:59.147202 17340 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 112 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:59.147584 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 112 candidate_status { last_received { term: 111 index: 2790 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:59.147688 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 111 FOLLOWER]: Advancing to term 112
I20250111 09:01:59.147739 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:01:59.147820 17340 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 113 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:59.147869 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 113 FOLLOWER]: Advancing to term 114
I20250111 09:01:59.148536 17474 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 112 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 112.
I20250111 09:01:59.148537 17340 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 114 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:59.148804 17340 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 114 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:59.148813 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 112 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:59.149032 18640 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 112 FOLLOWER]: Leader election won for term 112
I20250111 09:01:59.149209 18640 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 112 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:59.149268 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 114 candidate_status { last_received { term: 113 index: 2858 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:59.149362 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 113 FOLLOWER]: Advancing to term 114
I20250111 09:01:59.149333 18640 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2785, Committed index: 2785, Last appended: 111.2790, Last appended by leader: 2789, Current term: 112, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:59.150974 17467 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 112 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 111. Current term is 112. Ops: [111.2791-111.2791]
I20250111 09:01:59.151082 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 111 to 112, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 112 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:01:59.152874 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 112 candidate_status { last_received { term: 111 index: 2790 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:59.152976 17611 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 111 LEADER]: Stepping down as leader of term 111
I20250111 09:01:59.153041 17611 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 111 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:59.153149 17611 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2790, Committed index: 2790, Last appended: 111.2793, Last appended by leader: 2793, Current term: 111, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:59.153275 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 111 FOLLOWER]: Advancing to term 112
I20250111 09:01:59.154220 17611 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 112 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 112 because replica has last-logged OpId of term: 111 index: 2793, which is greater than that of the candidate, which has last-logged OpId of term: 111 index: 2790.
I20250111 09:01:59.154378 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 114 candidate_status { last_received { term: 113 index: 2858 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:59.154459 17613 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 113 LEADER]: Stepping down as leader of term 113
I20250111 09:01:59.154493 17613 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 113 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:59.154588 17613 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2858, Committed index: 2858, Last appended: 113.2858, Last appended by leader: 2858, Current term: 113, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:59.155481 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 113 FOLLOWER]: Advancing to term 114
I20250111 09:01:59.156307 17613 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 114 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 114.
I20250111 09:01:59.156944 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 114 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:01:59.157123 18640 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 114 FOLLOWER]: Leader election won for term 114
I20250111 09:01:59.157194 18640 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 114 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:59.157306 18640 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2858, Committed index: 2858, Last appended: 113.2858, Last appended by leader: 2858, Current term: 114, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:59.157861 18615 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: Replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 114 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
I20250111 09:01:59.159075 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 113 to 114, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 114 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:01:59.160805 18653 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.161753 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.162822 18650 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:59.163077 17474 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 114 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 114.
I20250111 09:01:59.163168 17467 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 114 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 113. Current term is 114. Ops: []
I20250111 09:01:59.163827 17467 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 114 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 113 index: 2858. Preceding OpId from leader: term: 114 index: 2860. (index mismatch)
I20250111 09:01:59.164297 17613 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 114 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 113 index: 2858. Preceding OpId from leader: term: 114 index: 2860. (index mismatch)
I20250111 09:01:59.164173 18657 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2859, Last known committed idx: 2854, Time since last communication: 0.000s
W20250111 09:01:59.165050 18634 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:59.165194 18640 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2859, Last known committed idx: 2858, Time since last communication: 0.000s
I20250111 09:01:59.171149 17467 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 112 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 111 index: 2790. Preceding OpId from leader: term: 112 index: 2792. (index mismatch)
W20250111 09:01:59.171973 18633 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:59.171965 18640 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2791, Last known committed idx: 2785, Time since last communication: 0.000s
I20250111 09:01:59.172457 17613 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 112 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 111 index: 2793. Preceding OpId from leader: term: 112 index: 2792. (term mismatch)
I20250111 09:01:59.172508 17613 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 2791
I20250111 09:01:59.172533 17613 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 111.2792
W20250111 09:01:59.172595 17613 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Aborted: Op aborted by new leader
I20250111 09:01:59.172714 17613 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 111.2793
W20250111 09:01:59.172796 17613 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:01:59.173381 18640 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2791, Last known committed idx: 2790, Time since last communication: 0.000s
I20250111 09:01:59.173465 18640 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 111.2791 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2790
I20250111 09:01:59.173908 17613 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 2790
I20250111 09:01:59.173949 17613 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 111.2791
W20250111 09:01:59.174018 17613 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:01:59.180546 18618 mvcc.cc:204] Tried to move back new op lower bound from 7113056744121528320 to 7113056744036773888. Current Snapshot: MvccSnapshot[applied={T|T < 7113056743973519360}]
W20250111 09:01:59.582033 18623 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 45 similar messages]
I20250111 09:01:59.861263 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:01:59.861740 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:59.863780 17476 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 112 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:59.863924 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 112 FOLLOWER]: Advancing to term 113
I20250111 09:01:59.868250 17476 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 113 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:59.868654 17476 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 113 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:59.868928 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 113 candidate_status { last_received { term: 112 index: 2966 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:59.869095 17336 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 112 LEADER]: Stepping down as leader of term 112
I20250111 09:01:59.869141 17336 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 112 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:59.869247 17336 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2968, Committed index: 2968, Last appended: 112.2968, Last appended by leader: 2968, Current term: 112, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:59.869374 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 112 FOLLOWER]: Advancing to term 113
I20250111 09:01:59.870484 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:01:59.871115 17336 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 113 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 113 because replica has last-logged OpId of term: 112 index: 2968, which is greater than that of the candidate, which has last-logged OpId of term: 112 index: 2966.
I20250111 09:01:59.868969 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 113 candidate_status { last_received { term: 112 index: 2966 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:59.871264 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 112 FOLLOWER]: Advancing to term 113
W20250111 09:01:59.872249 17283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:59.872417 17611 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 113 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 113 because replica has last-logged OpId of term: 112 index: 2968, which is greater than that of the candidate, which has last-logged OpId of term: 112 index: 2966.
I20250111 09:01:59.872916 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 113 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:59.873972 17476 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 114 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:59.874034 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 114 FOLLOWER]: Advancing to term 115
W20250111 09:01:59.874614 18660 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:59.877892 18666 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 113 FOLLOWER]: Leader election lost for term 113. Reason: could not achieve majority
I20250111 09:01:59.878459 17476 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 115 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:59.879518 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 115 candidate_status { last_received { term: 114 index: 3064 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:59.879552 17476 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 115 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:01:59.879609 17340 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 114 LEADER]: Stepping down as leader of term 114
I20250111 09:01:59.879643 17340 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 114 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:01:59.879741 17340 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3067, Committed index: 3067, Last appended: 114.3068, Last appended by leader: 3068, Current term: 114, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:01:59.880019 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.880064 18663 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.880913 18668 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 10 similar messages]
I20250111 09:01:59.881814 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 114 FOLLOWER]: Advancing to term 115
W20250111 09:01:59.882329 18668 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.881516 18667 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:59.882721 17340 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 115 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 115 because replica has last-logged OpId of term: 114 index: 3068, which is greater than that of the candidate, which has last-logged OpId of term: 114 index: 3064.
W20250111 09:01:59.883101 18619 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: Replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 115 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
I20250111 09:01:59.883781 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 115 candidate_status { last_received { term: 114 index: 3064 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:01:59.883874 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 114 FOLLOWER]: Advancing to term 115
W20250111 09:01:59.884805 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:59.885736 17613 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 115 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 115 because replica has last-logged OpId of term: 114 index: 3067, which is greater than that of the candidate, which has last-logged OpId of term: 114 index: 3064.
W20250111 09:01:59.885783 18667 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:59.885984 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 115 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:59.886183 18666 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 115 FOLLOWER]: Leader election lost for term 115. Reason: could not achieve majority
W20250111 09:01:59.887269 18670 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.888175 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.888290 18669 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.889360 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.889428 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.891422 18671 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 31 similar messages]
W20250111 09:01:59.891482 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.891726 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.892153 18673 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.892562 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.892901 18672 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.894539 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.894613 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.896142 18672 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.896595 18673 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.898041 18663 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.898808 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.899749 17283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.900272 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.901165 18660 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.901338 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.902092 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.902302 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.902339 18663 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.903719 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.904132 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.904616 18669 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.905604 18670 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.906004 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.906410 18663 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.907961 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.908720 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.910939 18670 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.911844 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.911933 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.912272 18669 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.913296 18673 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.913698 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.913879 18670 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.914119 18669 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.914510 18673 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.915863 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.916973 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.918809 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.919050 18673 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.920500 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.921198 18663 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.921660 18673 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.922649 18673 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.924057 18663 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.924396 18673 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.924448 18672 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.924631 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.925040 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.926064 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.927893 18669 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.928951 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.929383 18669 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.931440 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.931696 18669 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.933562 18663 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.934214 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.935504 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.935858 18673 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.936702 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.936841 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.936878 18663 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.938632 17283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.938869 18669 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.940296 18673 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.942452 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.944623 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.945732 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.945777 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:01:59.946619 18663 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.947132 18669 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.948182 18669 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.949826 18669 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.950650 18670 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.950663 18663 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.952142 18673 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:59.953449 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
W20250111 09:01:59.953495 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:59.953852 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:59.953954 17613 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 113 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:59.954015 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 113 FOLLOWER]: Advancing to term 114
I20250111 09:01:59.954717 17613 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 114 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:59.955018 17613 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 114 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:59.955262 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 114 candidate_status { last_received { term: 112 index: 2968 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:59.955278 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 114 candidate_status { last_received { term: 112 index: 2968 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:59.955356 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 113 FOLLOWER]: Advancing to term 114
I20250111 09:01:59.955390 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 113 FOLLOWER]: Advancing to term 114
I20250111 09:01:59.956104 17340 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 114 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 114.
I20250111 09:01:59.956126 17477 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 114 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 114.
W20250111 09:01:59.956255 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:01:59.956362 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 114 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:01:59.956462 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:01:59.956535 18654 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 114 FOLLOWER]: Leader election won for term 114
I20250111 09:01:59.956565 17613 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 115 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:01:59.956616 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 115 FOLLOWER]: Advancing to term 116
I20250111 09:01:59.956626 18654 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 114 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:59.956764 18654 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2968, Committed index: 2968, Last appended: 112.2968, Last appended by leader: 2968, Current term: 114, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:59.957405 17613 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 116 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:59.957731 17613 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 116 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:01:59.957985 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 116 candidate_status { last_received { term: 114 index: 3067 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:01:59.958076 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 115 FOLLOWER]: Advancing to term 116
I20250111 09:01:59.958109 17178 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 112 to 114, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 114 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:01:59.958906 17340 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 116 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 116 because replica has last-logged OpId of term: 114 index: 3068, which is greater than that of the candidate, which has last-logged OpId of term: 114 index: 3067.
I20250111 09:01:59.959808 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 116 candidate_status { last_received { term: 114 index: 3067 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:01:59.959929 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 115 FOLLOWER]: Advancing to term 116
W20250111 09:01:59.960214 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:59.961130 17467 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 114 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 112 index: 2966. Preceding OpId from leader: term: 114 index: 2970. (index mismatch)
I20250111 09:01:59.961195 17477 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 116 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 116.
I20250111 09:01:59.961503 18654 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2969, Last known committed idx: 2966, Time since last communication: 0.000s
I20250111 09:01:59.961516 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 116 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:01:59.961971 18654 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 116 FOLLOWER]: Leader election won for term 116
I20250111 09:01:59.962042 18654 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 116 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:01:59.962144 18654 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3064, Committed index: 3064, Last appended: 114.3067, Last appended by leader: 3067, Current term: 116, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:01:59.962466 17340 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 114 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 112 index: 2968. Preceding OpId from leader: term: 114 index: 2970. (index mismatch)
W20250111 09:01:59.962735 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:59.962904 18665 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2969, Last known committed idx: 2968, Time since last communication: 0.000s
W20250111 09:01:59.963240 18669 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.964550 18670 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.967980 17283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:01:59.971940 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 114 to 116, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 116 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
W20250111 09:01:59.972054 17283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.972630 17283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.976986 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.978346 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:01:59.979480 17336 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 116 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 114 index: 3068. Preceding OpId from leader: term: 116 index: 3069. (index mismatch)
I20250111 09:01:59.979828 18655 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3068, Last known committed idx: 3067, Time since last communication: 0.000s
I20250111 09:01:59.979928 18655 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer fb270e42d7c9400f8c05bb9565db1972 log is divergent from this leader: its last log entry 114.3068 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3067
I20250111 09:01:59.980515 17340 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 3067
I20250111 09:01:59.980559 17340 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted WRITE_OP operation due to leader change: 114.3068
W20250111 09:01:59.980638 17340 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Aborted: Op aborted by new leader
I20250111 09:01:59.982422 17477 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 116 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 114 index: 3064. Preceding OpId from leader: term: 116 index: 3069. (index mismatch)
I20250111 09:01:59.985901 18655 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3068, Last known committed idx: 3063, Time since last communication: 0.000s
W20250111 09:01:59.990496 18670 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.995347 18667 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.996099 18669 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:01:59.999233 18670 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:00.383421 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:02:00.384639 17336 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:00.384770 17336 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 114 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:00.384840 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 114 FOLLOWER]: Advancing to term 115
I20250111 09:02:00.385969 17336 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 115 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:00.386344 17336 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 115 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:00.387109 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 115 candidate_status { last_received { term: 114 index: 3087 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:00.387338 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 115 candidate_status { last_received { term: 114 index: 3087 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:00.387466 17613 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 114 LEADER]: Stepping down as leader of term 114
I20250111 09:02:00.387521 17613 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 114 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:00.387650 17613 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3087, Committed index: 3087, Last appended: 114.3087, Last appended by leader: 3087, Current term: 114, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:00.387741 17336 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 115 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 114. Current term is 115. Ops: []
I20250111 09:02:00.387890 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 114 FOLLOWER]: Advancing to term 115
I20250111 09:02:00.388054 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 114 FOLLOWER]: Advancing to term 115
W20250111 09:02:00.388330 18655 consensus_queue.cc:1159] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "262623ac6df745c785ce267ea39bd265" responder_term: 114 status { last_received { term: 114 index: 3087 } last_committed_idx: 3084 last_received_current_leader { term: 114 index: 3087 } }
I20250111 09:02:00.388546 17336 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
W20250111 09:02:00.388815 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:00.389045 17336 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 116 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:00.389073 17613 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 115 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 115.
I20250111 09:02:00.389099 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 116 FOLLOWER]: Advancing to term 117
W20250111 09:02:00.389528 18614 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: Replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 115 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
I20250111 09:02:00.389891 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 115 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:00.390044 18657 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 115 FOLLOWER]: Leader election won for term 115
I20250111 09:02:00.390185 18657 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 115 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:00.390304 18657 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3084, Committed index: 3084, Last appended: 114.3087, Last appended by leader: 3087, Current term: 115, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:00.390462 18614 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.005s ago.
W20250111 09:02:00.390853 18614 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.005s ago.
I20250111 09:02:00.389906 17336 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 117 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:00.391777 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 114 to 115, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 115 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:00.392562 17467 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 115 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 115.
I20250111 09:02:00.392572 17336 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 117 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:00.393219 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 117 candidate_status { last_received { term: 116 index: 3161 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:00.393321 17613 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 116 LEADER]: Stepping down as leader of term 116
I20250111 09:02:00.393381 17613 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 116 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:00.393502 17613 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3160, Committed index: 3160, Last appended: 116.3163, Last appended by leader: 3163, Current term: 116, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:00.393643 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 116 FOLLOWER]: Advancing to term 117
I20250111 09:02:00.394625 17613 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 117 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 117 because replica has last-logged OpId of term: 116 index: 3163, which is greater than that of the candidate, which has last-logged OpId of term: 116 index: 3161.
I20250111 09:02:00.395589 17613 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 115 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 114 index: 3087. Preceding OpId from leader: term: 115 index: 3089. (index mismatch)
I20250111 09:02:00.395932 18683 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3088, Last known committed idx: 3087, Time since last communication: 0.000s
I20250111 09:02:00.391351 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 117 candidate_status { last_received { term: 116 index: 3161 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:00.396364 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 116 FOLLOWER]: Advancing to term 117
I20250111 09:02:00.397408 17477 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 117 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 117.
I20250111 09:02:00.397682 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 117 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters: b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:00.397886 17476 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 115 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 114 index: 3087. Preceding OpId from leader: term: 115 index: 3089. (index mismatch)
W20250111 09:02:00.398272 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:00.398669 18683 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 117 FOLLOWER]: Leader election won for term 117
W20250111 09:02:00.398782 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:00.398836 18683 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 117 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:00.398947 18683 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3160, Committed index: 3160, Last appended: 116.3161, Last appended by leader: 3161, Current term: 117, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:00.400740 18657 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3088, Last known committed idx: 3084, Time since last communication: 0.000s
I20250111 09:02:00.400934 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 116 to 117, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 117 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:00.402690 17474 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 117 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 116 index: 3161. Preceding OpId from leader: term: 117 index: 3163. (index mismatch)
I20250111 09:02:00.402845 17613 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 117 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 116 index: 3163. Preceding OpId from leader: term: 117 index: 3163. (term mismatch)
I20250111 09:02:00.402899 17613 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 3162
I20250111 09:02:00.402944 17613 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 116.3163
W20250111 09:02:00.403028 17613 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:02:00.403441 18657 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3162, Last known committed idx: 3160, Time since last communication: 0.000s
I20250111 09:02:00.404528 18657 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3162, Last known committed idx: 3160, Time since last communication: 0.000s
I20250111 09:02:00.404624 18657 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 116.3162 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3160
I20250111 09:02:00.405716 17613 raft_consensus.cc:1212] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 117 FOLLOWER]: Deduplicated request from leader. Original: 116.3160->[116.3161-117.3163] Dedup: 116.3161->[117.3162-117.3163]
I20250111 09:02:00.405795 17613 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 3161
I20250111 09:02:00.405820 17613 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 116.3162
W20250111 09:02:00.405896 17613 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:02:00.411027 18614 mvcc.cc:204] Tried to move back new op lower bound from 7113056749166080000 to 7113056749155577856. Current Snapshot: MvccSnapshot[applied={T|T < 7113056749166080000}]
W20250111 09:02:00.630695 18623 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 57 similar messages]
I20250111 09:02:00.790915 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:02:00.791473 17473 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:00.791586 17473 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 115 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:00.791637 17473 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 115 FOLLOWER]: Advancing to term 116
I20250111 09:02:00.792629 17473 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 116 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:00.793960 17476 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 116 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 115. Current term is 116. Ops: [115.3176-115.3176]
I20250111 09:02:00.793959 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 116 candidate_status { last_received { term: 115 index: 3175 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:00.794095 17336 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 115 LEADER]: Stepping down as leader of term 115
I20250111 09:02:00.794140 17336 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 115 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:00.794252 17336 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3175, Committed index: 3175, Last appended: 115.3176, Last appended by leader: 3176, Current term: 115, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:00.794358 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 115 FOLLOWER]: Advancing to term 116
I20250111 09:02:00.795246 17336 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 116 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 116 because replica has last-logged OpId of term: 115 index: 3176, which is greater than that of the candidate, which has last-logged OpId of term: 115 index: 3175.
W20250111 09:02:00.795596 17299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:00.796007 17473 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 116 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:00.796083 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 116 candidate_status { last_received { term: 115 index: 3175 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:00.796191 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 115 FOLLOWER]: Advancing to term 116
I20250111 09:02:00.796924 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:00.797215 17611 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 116 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 116 because replica has last-logged OpId of term: 115 index: 3176, which is greater than that of the candidate, which has last-logged OpId of term: 115 index: 3175.
I20250111 09:02:00.797475 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 116 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:00.797645 18666 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 116 FOLLOWER]: Leader election lost for term 116. Reason: could not achieve majority
W20250111 09:02:00.801178 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:00.801275 17476 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 117 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:00.801329 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 117 FOLLOWER]: Advancing to term 118
I20250111 09:02:00.802419 17476 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 118 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:00.802752 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:00.802783 17474 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 118 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 117. Current term is 118. Ops: [117.3254-117.3256]
I20250111 09:02:00.802951 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 118 candidate_status { last_received { term: 117 index: 3253 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
W20250111 09:02:00.803031 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:00.803035 17336 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 117 LEADER]: Stepping down as leader of term 117
I20250111 09:02:00.803081 17336 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 117 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:00.803181 17336 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3253, Committed index: 3253, Last appended: 117.3257, Last appended by leader: 3257, Current term: 117, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:00.803267 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 117 FOLLOWER]: Advancing to term 118
I20250111 09:02:00.803995 17336 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 118 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 118 because replica has last-logged OpId of term: 117 index: 3257, which is greater than that of the candidate, which has last-logged OpId of term: 117 index: 3253.
W20250111 09:02:00.805804 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:00.806311 17476 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 118 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
W20250111 09:02:00.806866 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:00.806926 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 118 candidate_status { last_received { term: 117 index: 3253 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:00.807102 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 117 FOLLOWER]: Advancing to term 118
I20250111 09:02:00.808116 17613 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 118 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 118 because replica has last-logged OpId of term: 117 index: 3256, which is greater than that of the candidate, which has last-logged OpId of term: 117 index: 3253.
I20250111 09:02:00.808410 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 118 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:00.808557 18666 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 118 FOLLOWER]: Leader election lost for term 118. Reason: could not achieve majority
W20250111 09:02:00.809291 18696 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.811172 18696 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.812774 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.815526 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.815222 18696 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.817203 18673 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.817296 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.818529 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.823122 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.824342 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.824633 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.825493 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.827797 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.830928 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.832230 18696 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.833057 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.833863 18696 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.836755 18696 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.839747 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.841231 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.841614 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.842319 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.844185 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.848933 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.849541 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.853196 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.853668 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.854710 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.858099 18696 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.861320 18696 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.862494 18696 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.863672 18696 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.866050 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.870944 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.872524 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.875995 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.877449 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.879503 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.879738 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.886559 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.886827 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.888763 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.892280 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.894117 18696 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 35 similar messages]
W20250111 09:02:00.894169 18696 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.899569 18696 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.899578 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.903613 18696 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.905149 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.907344 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.913365 18671 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 69 similar messages]
W20250111 09:02:00.913411 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.916044 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.916095 18673 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.921275 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.923147 18673 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.929796 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.932873 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.933321 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.937448 18696 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.939010 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.945154 18696 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.948102 18696 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.949450 18696 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.952765 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.953739 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.966045 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.966150 18673 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.967841 18673 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.968197 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:00.971338 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.985409 18696 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.986721 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.988072 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.989341 17299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:00.991243 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.005271 18696 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.006279 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.007998 18696 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.008047 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.012950 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.024861 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.026335 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.029109 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.030543 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.032588 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.046590 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.046713 17299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.050012 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.052382 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.052714 17299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.070472 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.071300 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.073945 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.075160 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.077474 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.093855 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.095501 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.098243 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.098752 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.099527 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.118355 17299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.119463 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.121642 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.121642 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.125119 17299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.143555 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.145661 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.145867 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.149380 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.150517 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.169229 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.170346 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.171779 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.177479 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.179721 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.196267 17299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.197692 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.198680 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.206763 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.207793 17299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.224743 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.226815 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.228216 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.234480 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.236881 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.254061 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.255553 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.257068 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.265228 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.266800 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:01.271788 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:02:01.272199 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:01.272285 17613 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 116 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:01.272346 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 116 FOLLOWER]: Advancing to term 117
I20250111 09:02:01.273094 17613 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 117 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:01.273530 17613 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 117 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:01.273834 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 117 candidate_status { last_received { term: 115 index: 3176 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:01.273949 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 116 FOLLOWER]: Advancing to term 117
I20250111 09:02:01.273986 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 117 candidate_status { last_received { term: 115 index: 3176 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:01.274071 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 116 FOLLOWER]: Advancing to term 117
I20250111 09:02:01.274683 17476 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 117 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 117.
I20250111 09:02:01.274844 17336 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 117 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 117.
I20250111 09:02:01.274914 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 117 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:02:01.275110 18655 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 117 FOLLOWER]: Leader election won for term 117
I20250111 09:02:01.275197 18655 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 117 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:01.275315 18655 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3175, Committed index: 3175, Last appended: 115.3176, Last appended by leader: 3176, Current term: 117, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:01.275477 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:01.275566 17613 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 118 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:01.275610 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 118 FOLLOWER]: Advancing to term 119
I20250111 09:02:01.276429 17613 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 119 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:01.276530 17178 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 115 to 117, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 117 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:01.276728 17613 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 119 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:01.276949 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 119 candidate_status { last_received { term: 117 index: 3256 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:01.277073 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 118 FOLLOWER]: Advancing to term 119
I20250111 09:02:01.277204 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 119 candidate_status { last_received { term: 117 index: 3256 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:01.277308 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 118 FOLLOWER]: Advancing to term 119
I20250111 09:02:01.277904 17336 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 119 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 119 because replica has last-logged OpId of term: 117 index: 3257, which is greater than that of the candidate, which has last-logged OpId of term: 117 index: 3256.
I20250111 09:02:01.278088 17476 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 119 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 119.
I20250111 09:02:01.278307 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 119 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:01.278491 18655 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 119 FOLLOWER]: Leader election won for term 119
I20250111 09:02:01.278566 18655 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 119 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:01.278671 18655 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3253, Committed index: 3253, Last appended: 117.3256, Last appended by leader: 3256, Current term: 119, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:01.279753 17178 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 117 to 119, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 119 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:01.285115 17476 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 119 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 117 index: 3253. Preceding OpId from leader: term: 119 index: 3258. (index mismatch)
I20250111 09:02:01.285125 17336 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 119 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 117 index: 3257. Preceding OpId from leader: term: 119 index: 3258. (index mismatch)
I20250111 09:02:01.285485 18665 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3257, Last known committed idx: 3253, Time since last communication: 0.000s
I20250111 09:02:01.285576 18665 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer fb270e42d7c9400f8c05bb9565db1972 log is divergent from this leader: its last log entry 117.3257 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3253
I20250111 09:02:01.285662 18655 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3257, Last known committed idx: 3251, Time since last communication: 0.000s
I20250111 09:02:01.286278 17336 raft_consensus.cc:1212] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 119 FOLLOWER]: Deduplicated request from leader. Original: 117.3253->[117.3254-119.3258] Dedup: 117.3256->[119.3257-119.3258]
I20250111 09:02:01.286335 17336 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 3256
I20250111 09:02:01.286350 17336 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 117.3257
W20250111 09:02:01.286415 17336 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Aborted: Op aborted by new leader
W20250111 09:02:01.290036 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:01.292906 18597 mvcc.cc:204] Tried to move back new op lower bound from 7113056752781021184 to 7113056752758575104. Current Snapshot: MvccSnapshot[applied={T|T < 7113056752781021184}]
I20250111 09:02:01.295060 17476 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 117 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 115 index: 3175. Preceding OpId from leader: term: 117 index: 3179. (index mismatch)
W20250111 09:02:01.296290 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:01.296358 18665 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3177, Last known committed idx: 3174, Time since last communication: 0.000s
I20250111 09:02:01.297186 17336 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 117 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 115 index: 3176. Preceding OpId from leader: term: 117 index: 3179. (index mismatch)
I20250111 09:02:01.298620 18665 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3177, Last known committed idx: 3175, Time since last communication: 0.000s
W20250111 09:02:01.320585 18704 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.327174 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.334439 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:01.489147 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:02:01.489895 17335 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:01.494566 17335 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 117 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:01.494711 17335 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 117 FOLLOWER]: Advancing to term 118
I20250111 09:02:01.496693 17335 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 118 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:01.498267 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 118 candidate_status { last_received { term: 117 index: 3230 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:01.498987 17335 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 118 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:01.499711 17336 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:01.499806 17336 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 119 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:01.499883 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 119 FOLLOWER]: Advancing to term 120
I20250111 09:02:01.500725 17336 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 120 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:01.503224 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 120 candidate_status { last_received { term: 119 index: 3304 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:01.505481 17336 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 120 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:01.513105 17335 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 118 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 117. Current term is 118. Ops: [117.3231-117.3238]
I20250111 09:02:01.513396 17336 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 120 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 119. Current term is 120. Ops: [119.3305-119.3314]
I20250111 09:02:01.513878 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 118 candidate_status { last_received { term: 117 index: 3230 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:01.513990 17613 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 117 LEADER]: Stepping down as leader of term 117
I20250111 09:02:01.514032 17613 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 117 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:01.514174 17613 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3239, Committed index: 3239, Last appended: 117.3239, Last appended by leader: 3239, Current term: 117, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:01.514307 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 117 FOLLOWER]: Advancing to term 118
I20250111 09:02:01.515295 17613 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 118 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 118 because replica has last-logged OpId of term: 117 index: 3239, which is greater than that of the candidate, which has last-logged OpId of term: 117 index: 3230.
I20250111 09:02:01.515913 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 120 candidate_status { last_received { term: 119 index: 3304 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:01.515998 17613 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 119 LEADER]: Stepping down as leader of term 119
I20250111 09:02:01.516031 17613 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 119 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:01.516129 17613 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3319, Committed index: 3319, Last appended: 119.3319, Last appended by leader: 3319, Current term: 119, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:01.516213 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:01.516213 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 119 FOLLOWER]: Advancing to term 120
I20250111 09:02:01.517120 17613 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 120 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 120 because replica has last-logged OpId of term: 119 index: 3319, which is greater than that of the candidate, which has last-logged OpId of term: 119 index: 3304.
I20250111 09:02:01.517776 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 118 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:01.518030 18690 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 118 FOLLOWER]: Leader election lost for term 118. Reason: could not achieve majority
W20250111 09:02:01.518241 18614 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: Replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 120 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
W20250111 09:02:01.519098 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.519155 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.520521 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.520576 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:01.520622 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 120 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:01.520812 18690 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 120 FOLLOWER]: Leader election lost for term 120. Reason: could not achieve majority
W20250111 09:02:01.523981 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.524236 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.525054 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.527667 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.528564 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.528630 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.529145 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.530328 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.531004 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.531422 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.531962 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.532572 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.532584 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.532915 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.535563 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.535773 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.535802 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.536410 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.536901 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.538877 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.539592 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.540084 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.540141 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.540179 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.540390 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.542804 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.543713 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.544005 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.544189 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.543170 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.548897 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.549557 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.549643 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.549974 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.550216 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.550909 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.553465 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.553658 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.554049 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.554085 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.554329 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.554813 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.554857 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.555059 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.556420 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.556517 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.557787 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.558646 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.561312 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.561918 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.562234 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.562844 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.563285 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.563526 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.563540 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.563563 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.564380 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.564426 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.565003 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.565171 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.565672 18718 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.568428 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.569578 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.570686 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.570947 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.571331 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.571903 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.572223 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.573050 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.573155 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.573578 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.574215 18718 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.575198 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.576761 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.578140 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.578166 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.578716 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.578946 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.579618 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.580912 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.580927 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.581786 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.582039 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.583026 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.583122 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.584522 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.586002 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.586019 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.586898 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.588783 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.588804 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.589030 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.590199 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.590281 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.590615 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.591953 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.592111 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.593847 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.594852 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.595855 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.596439 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.597208 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.598999 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.599442 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.600314 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.600919 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.601286 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.601490 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.601523 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.602562 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.605728 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.606689 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.607504 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.609097 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.609556 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.609655 18718 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.609963 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.611158 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.613303 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.613595 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.613833 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.614595 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.616142 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.617069 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.617265 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.619000 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.620556 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.622026 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.622109 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.622470 18718 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.623136 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.625558 18718 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.628286 17315 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.628427 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.628468 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.628985 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.630295 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.631479 18718 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.631834 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.634359 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.634979 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.635099 18716 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 61 similar messages]
W20250111 09:02:01.635161 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.637501 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.637684 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.639389 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.639935 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.641633 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.642772 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.642867 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.643597 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.644985 18718 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.646479 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.647615 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.649670 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.649750 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.651016 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.652122 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.652410 18718 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.652431 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.655547 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.656457 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.657140 18718 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.659777 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.660192 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.660831 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.662528 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.663995 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.664263 18718 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.664717 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.665194 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.667750 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.668869 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.669843 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.670977 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.671993 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.674189 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.674536 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.678270 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.678941 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.679258 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.679647 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.679672 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.684082 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.684238 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.684687 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.685948 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.687245 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.688700 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.689131 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.690091 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.693235 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.695457 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.696043 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.696106 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.696237 18718 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.697046 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.699805 18718 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.703478 18718 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.703517 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.705740 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.706104 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.706377 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.707525 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.708262 18718 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.712812 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.714310 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.715083 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.715560 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.715759 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.715901 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.717515 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.723423 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.723495 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.725056 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.725524 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.725809 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.726094 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.727298 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.732403 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.733479 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.734108 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.734328 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.735749 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.738231 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.738436 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.744834 18671 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.745101 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.745342 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.746526 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.747542 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.747921 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.748625 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.751535 18718 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.752758 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.753376 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.755048 18718 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.756199 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.760841 18718 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.761013 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.765975 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.767294 18717 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.768239 18718 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.768278 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.769132 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.769294 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.770447 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.772055 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.774806 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.774955 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.776911 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.777149 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.780303 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.783246 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.785980 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.788985 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.789129 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.790634 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.791754 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.792176 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.792703 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.793922 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.798151 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.798892 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.800299 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.800529 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.802732 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.804034 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.808315 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.809760 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.810149 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.814306 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.816182 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.816983 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.817370 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.818389 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.820523 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.824015 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.824731 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.825723 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.826352 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.827165 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.831729 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.833202 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.834626 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.838908 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.840791 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.841526 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.844300 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.844399 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.846611 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.847332 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.850731 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.850960 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.852162 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.853758 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.857954 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.858942 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.859059 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.863694 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.864041 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.867215 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.868309 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.870293 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.871542 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.872704 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.877257 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.877902 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.878075 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.880050 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.883255 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.883525 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.885536 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.891353 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.891343 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.894165 18697 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 80 similar messages]
W20250111 09:02:01.894222 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.894471 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.894551 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.895679 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.898998 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.901978 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.904556 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.906246 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.908360 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.909619 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.910900 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.911192 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.919252 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.920595 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.921633 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.921833 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.922830 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.924178 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.925745 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.929813 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.930524 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.936061 18706 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 69 similar messages]
W20250111 09:02:01.936108 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.936115 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.936157 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.936581 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.940402 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.947374 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.948607 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.948695 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.949728 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.951905 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.951972 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.953876 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.959347 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.960006 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.963749 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.964864 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.966269 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.967383 18706 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:01.970104 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.977396 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.977466 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.978619 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.978608 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.982731 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.983469 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.984798 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.988970 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:01.990558 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:01.992992 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:02:01.993448 17473 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:01.993569 17473 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 117 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:01.993624 17473 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 117 FOLLOWER]: Advancing to term 118
W20250111 09:02:01.994436 18718 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:01.994693 17473 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 118 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:01.994922 18697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:01.995080 17473 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 118 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:01.995522 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 118 candidate_status { last_received { term: 117 index: 3239 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:01.995656 17336 raft_consensus.cc:2388] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 118 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 in current term 118: Already voted for candidate fb270e42d7c9400f8c05bb9565db1972 in this term.
I20250111 09:02:01.995602 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 118 candidate_status { last_received { term: 117 index: 3239 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:01.996151 17473 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:01.996248 17473 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 119 FOLLOWER]: Starting forced leader election (received explicit request)
W20250111 09:02:01.996254 18718 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:01.996315 17473 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 119 FOLLOWER]: Advancing to term 120
I20250111 09:02:01.996661 17613 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 118 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 118.
I20250111 09:02:01.996901 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 118 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:01.997220 17473 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 120 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:01.997282 18719 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 118 FOLLOWER]: Leader election won for term 118
I20250111 09:02:01.997423 18719 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 118 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:01.997566 18719 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3239, Committed index: 3239, Last appended: 117.3239, Last appended by leader: 3239, Current term: 118, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:01.997778 17473 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 120 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:01.997949 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 120 candidate_status { last_received { term: 119 index: 3319 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:01.998077 17336 raft_consensus.cc:2388] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 120 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 in current term 120: Already voted for candidate fb270e42d7c9400f8c05bb9565db1972 in this term.
I20250111 09:02:01.998253 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 120 candidate_status { last_received { term: 119 index: 3319 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
W20250111 09:02:01.999020 18718 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:01.999174 17613 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 120 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 120.
I20250111 09:02:01.999267 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 117 to 118, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 118 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:01.999666 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 120 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:01.999876 18719 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 120 FOLLOWER]: Leader election won for term 120
I20250111 09:02:01.999971 18719 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 120 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:02.000092 18719 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3319, Committed index: 3319, Last appended: 119.3319, Last appended by leader: 3319, Current term: 120, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:02.001106 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:02.002427 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 119 to 120, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 120 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:02.009402 17613 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 120 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 119 index: 3319. Preceding OpId from leader: term: 120 index: 3321. (index mismatch)
I20250111 09:02:02.009406 17336 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 120 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 119 index: 3304. Preceding OpId from leader: term: 120 index: 3321. (index mismatch)
I20250111 09:02:02.009738 18719 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3320, Last known committed idx: 3319, Time since last communication: 0.000s
I20250111 09:02:02.011011 18725 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3320, Last known committed idx: 3303, Time since last communication: 0.000s
I20250111 09:02:02.011314 17613 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 118 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 117 index: 3239. Preceding OpId from leader: term: 118 index: 3241. (index mismatch)
I20250111 09:02:02.011366 17336 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 118 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 117 index: 3230. Preceding OpId from leader: term: 118 index: 3241. (index mismatch)
I20250111 09:02:02.011636 18720 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3240, Last known committed idx: 3239, Time since last communication: 0.000s
I20250111 09:02:02.011866 18720 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3240, Last known committed idx: 3226, Time since last communication: 0.000s
W20250111 09:02:02.024991 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:02.027199 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:02.028573 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:02.450402 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:02:02.451094 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:02.451232 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 118 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:02.451303 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 118 FOLLOWER]: Advancing to term 119
I20250111 09:02:02.452605 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 119 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:02.453088 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 119 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:02.453289 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 119 candidate_status { last_received { term: 118 index: 3342 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:02.453423 17335 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 118 FOLLOWER]: Advancing to term 119
I20250111 09:02:02.454066 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:02.454185 17611 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 120 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:02.454241 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 120 FOLLOWER]: Advancing to term 121
I20250111 09:02:02.454080 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 119 candidate_status { last_received { term: 118 index: 3342 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:02.454541 17476 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 118 LEADER]: Stepping down as leader of term 118
I20250111 09:02:02.454604 17476 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 118 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:02.454756 17335 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 119 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 119.
I20250111 09:02:02.454748 17476 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3340, Committed index: 3340, Last appended: 118.3343, Last appended by leader: 3343, Current term: 118, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:02.454844 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 118 FOLLOWER]: Advancing to term 119
I20250111 09:02:02.454998 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 119 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:02.455116 17611 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 121 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:02.455250 18715 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 119 FOLLOWER]: Leader election won for term 119
I20250111 09:02:02.455439 17611 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 121 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:02.455718 17476 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 119 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 119 because replica has last-logged OpId of term: 118 index: 3343, which is greater than that of the candidate, which has last-logged OpId of term: 118 index: 3342.
I20250111 09:02:02.455718 17611 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 121 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 120. Current term is 121. Ops: []
I20250111 09:02:02.456035 18725 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: INVALID_TERM, Last received: 120.3431, Next index: 3432, Last known committed idx: 3429, Time since last communication: 0.000s
I20250111 09:02:02.456110 18715 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 119 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
W20250111 09:02:02.456218 18591 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.007s ago.
I20250111 09:02:02.456256 18719 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 120 LEADER]: Stepping down as leader of term 120
I20250111 09:02:02.456300 18719 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 120 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:02.456416 18719 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3431, Committed index: 3431, Last appended: 120.3431, Last appended by leader: 3431, Current term: 120, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:02.456444 18715 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3340, Committed index: 3340, Last appended: 118.3342, Last appended by leader: 3342, Current term: 119, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:02.456629 18719 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 120 FOLLOWER]: Advancing to term 121
I20250111 09:02:02.457319 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 121 candidate_status { last_received { term: 120 index: 3431 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:02.457429 17335 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 120 FOLLOWER]: Advancing to term 121
W20250111 09:02:02.457912 18591 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: Replica 262623ac6df745c785ce267ea39bd265 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 121 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
I20250111 09:02:02.458046 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 118 to 119, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 119 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:02.458678 17335 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 121 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 121.
I20250111 09:02:02.458914 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 121 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:02.458993 17340 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 119 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 118. Current term is 119. Ops: [118.3341-118.3342]
I20250111 09:02:02.459093 18715 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 121 FOLLOWER]: Leader election won for term 121
I20250111 09:02:02.459216 18715 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 121 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:02.458956 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 121 candidate_status { last_received { term: 120 index: 3431 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:02.459319 18715 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3429, Committed index: 3429, Last appended: 120.3431, Last appended by leader: 3431, Current term: 121, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:02.460218 17340 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 121 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 120. Current term is 121. Ops: [120.3429-120.3429]
I20250111 09:02:02.461141 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 120 to 121, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 121 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:02.461896 17476 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 121 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 121.
W20250111 09:02:02.462404 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:02.462404 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:02.462553 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:02.463179 17434 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:02.467154 18716 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:02.467494 18688 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:02.467952 18737 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:02.468097 18736 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:02.469592 18737 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:02.469919 18736 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:02.474679 17340 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 121 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 120 index: 3428. Preceding OpId from leader: term: 121 index: 3433. (index mismatch)
I20250111 09:02:02.474809 17476 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 121 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 120 index: 3431. Preceding OpId from leader: term: 121 index: 3433. (index mismatch)
I20250111 09:02:02.475252 18734 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3432, Last known committed idx: 3431, Time since last communication: 0.000s
I20250111 09:02:02.475459 18734 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3432, Last known committed idx: 3425, Time since last communication: 0.000s
I20250111 09:02:02.477300 17473 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 119 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 118 index: 3343. Preceding OpId from leader: term: 119 index: 3344. (index mismatch)
I20250111 09:02:02.481063 18715 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3343, Last known committed idx: 3340, Time since last communication: 0.000s
I20250111 09:02:02.481155 18715 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer 262623ac6df745c785ce267ea39bd265 log is divergent from this leader: its last log entry 118.3343 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3340
I20250111 09:02:02.481513 17337 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 119 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 118 index: 3340. Preceding OpId from leader: term: 119 index: 3344. (index mismatch)
I20250111 09:02:02.481848 18735 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3343, Last known committed idx: 3339, Time since last communication: 0.000s
I20250111 09:02:02.481886 17473 raft_consensus.cc:1212] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 119 FOLLOWER]: Deduplicated request from leader. Original: 118.3340->[118.3341-119.3345] Dedup: 118.3342->[119.3343-119.3345]
I20250111 09:02:02.481950 17473 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 3342
I20250111 09:02:02.481971 17473 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 118.3343
W20250111 09:02:02.482036 17473 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
W20250111 09:02:02.500432 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:02.500509 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:02.503414 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:02.503422 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:02.941844 18614 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 50 similar messages]
I20250111 09:02:03.396848 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:02:03.397536 17337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:03.399286 17337 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 119 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:03.399367 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 119 FOLLOWER]: Advancing to term 120
I20250111 09:02:03.402416 17337 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 120 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:03.402904 17337 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 120 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:03.403259 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 120 candidate_status { last_received { term: 119 index: 3576 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:03.403407 17337 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 120 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 119. Current term is 120. Ops: [119.3577-119.3579]
I20250111 09:02:03.403443 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:03.403439 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 119 FOLLOWER]: Advancing to term 120
I20250111 09:02:03.404439 17476 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 120 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 120.
I20250111 09:02:03.404719 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 120 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:03.404804 18743 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: INVALID_TERM, Last received: 119.3576, Next index: 3577, Last known committed idx: 3575, Time since last communication: 0.000s
I20250111 09:02:03.404897 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 120 candidate_status { last_received { term: 119 index: 3576 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:03.404948 18743 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 119 LEADER]: Stepping down as leader of term 119
I20250111 09:02:03.405045 18743 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 119 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:03.405190 18743 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3576, Committed index: 3576, Last appended: 119.3579, Last appended by leader: 3579, Current term: 119, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:03.405361 18743 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 119 FOLLOWER]: Advancing to term 120
I20250111 09:02:03.406109 18749 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 120 FOLLOWER]: Leader election won for term 120
I20250111 09:02:03.406304 18749 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 120 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:03.406455 18749 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3575, Committed index: 3575, Last appended: 119.3576, Last appended by leader: 3576, Current term: 120, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:03.407181 17613 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 120 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 120 because replica has last-logged OpId of term: 119 index: 3579, which is greater than that of the candidate, which has last-logged OpId of term: 119 index: 3576.
I20250111 09:02:03.411645 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 119 to 120, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 120 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:03.412148 17340 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 121 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:03.412227 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 121 FOLLOWER]: Advancing to term 122
I20250111 09:02:03.413142 17340 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 122 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:03.413556 17340 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 122 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:03.414040 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 122 candidate_status { last_received { term: 121 index: 3670 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:03.414620 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 122 candidate_status { last_received { term: 121 index: 3670 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:03.414706 17613 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 121 LEADER]: Stepping down as leader of term 121
I20250111 09:02:03.414752 17613 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 121 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:03.414862 17613 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3671, Committed index: 3671, Last appended: 121.3673, Last appended by leader: 3673, Current term: 121, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:03.414973 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 121 FOLLOWER]: Advancing to term 122
I20250111 09:02:03.415841 17613 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 122 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 122 because replica has last-logged OpId of term: 121 index: 3673, which is greater than that of the candidate, which has last-logged OpId of term: 121 index: 3670.
I20250111 09:02:03.416178 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 122 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:03.416318 18749 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 122 FOLLOWER]: Leader election lost for term 122. Reason: could not achieve majority
W20250111 09:02:03.416782 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.417551 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.425236 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.425405 18754 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 52 similar messages]
W20250111 09:02:03.425472 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.425693 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.426004 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.426821 18740 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:03.428018 17613 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 120 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 119 index: 3579. Preceding OpId from leader: term: 120 index: 3578. (term mismatch)
I20250111 09:02:03.428092 17613 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 3577
I20250111 09:02:03.428112 17613 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 119.3578
W20250111 09:02:03.428190 17613 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Aborted: Op aborted by new leader
I20250111 09:02:03.428333 17613 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 119.3579
W20250111 09:02:03.428427 17613 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:02:03.428825 17474 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 120 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 119 index: 3576. Preceding OpId from leader: term: 120 index: 3578. (index mismatch)
I20250111 09:02:03.428901 18750 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3577, Last known committed idx: 3576, Time since last communication: 0.000s
I20250111 09:02:03.429013 18750 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 119.3577 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3576
I20250111 09:02:03.429190 18750 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3577, Last known committed idx: 3575, Time since last communication: 0.000s
I20250111 09:02:03.429524 17613 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 3576
I20250111 09:02:03.429574 17613 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 119.3577
W20250111 09:02:03.429669 17613 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
W20250111 09:02:03.431565 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.437770 18759 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 55 similar messages]
W20250111 09:02:03.437880 18759 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.440718 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.441948 18759 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.442618 18758 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.446337 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.449791 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.449838 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.451565 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.454473 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.455755 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.458551 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.460984 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.461612 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.463130 18758 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.465267 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.466393 18758 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.468164 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.469153 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.470475 18759 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.471540 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.472478 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.474398 18740 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.475564 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.476368 18759 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.477303 18758 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.477913 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.480072 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.480116 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.482214 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.485167 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.485710 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.486622 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.486845 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.487043 18758 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.489075 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.492731 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.492785 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.497265 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.497591 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.497972 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.500133 18758 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.501394 18758 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.503346 18758 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.504698 18758 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.504851 18759 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.506256 18759 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.508111 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.512560 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.512852 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.515414 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.516497 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.516671 18740 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.520416 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.521090 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.525046 18740 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.525439 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.526782 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.529405 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.529456 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.530858 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.535887 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.536664 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.540565 18759 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.540614 18758 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.542179 18761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.542371 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.546069 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.547822 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.553586 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.554376 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.556531 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.558046 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.559252 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.562824 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.562824 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.567624 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.568753 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.572203 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.574872 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.576069 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.576809 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.577481 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.580981 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.585155 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.589906 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.592200 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.593603 18761 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.593679 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.593686 18758 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.598336 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.603543 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.607933 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.611296 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.613106 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.613899 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.615001 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.615348 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.623134 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.626538 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.632735 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.633170 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.633564 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.634104 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.634657 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.641350 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.645449 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.651540 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.651989 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.652123 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.654706 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.655959 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.659196 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.663448 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.673183 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.674805 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.675546 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.676316 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.678994 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.681264 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.681524 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.694303 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.695309 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.699072 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.701112 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.701202 18758 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.702294 18758 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.703073 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.716156 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.717845 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.723758 18755 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.723871 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.724486 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.726737 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.728148 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.739032 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.741839 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.747004 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.748405 18758 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.750877 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.751176 18758 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.751799 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.762943 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.765981 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.771766 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.773756 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.775753 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.776175 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.779654 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.786715 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.793660 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.797533 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.798300 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.799821 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.803400 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.807140 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.813820 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.817991 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.823586 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.824640 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.824685 18758 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.831779 18758 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.837440 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.841560 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.843057 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.849356 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.852486 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:03.853034 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
W20250111 09:02:03.853431 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:03.853428 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:03.853528 17476 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 120 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:03.853586 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 120 FOLLOWER]: Advancing to term 121
I20250111 09:02:03.854305 17476 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 121 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:03.854641 17476 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 121 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:03.854876 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 121 candidate_status { last_received { term: 120 index: 3591 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:03.854915 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 121 candidate_status { last_received { term: 120 index: 3591 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:03.854992 17336 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 120 LEADER]: Stepping down as leader of term 120
I20250111 09:02:03.855012 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 120 FOLLOWER]: Advancing to term 121
I20250111 09:02:03.855037 17336 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 120 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:03.855168 17336 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3591, Committed index: 3591, Last appended: 120.3591, Last appended by leader: 3591, Current term: 120, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:03.855266 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:03.855309 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 120 FOLLOWER]: Advancing to term 121
I20250111 09:02:03.855343 17476 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 121 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:03.855393 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 121 FOLLOWER]: Advancing to term 122
I20250111 09:02:03.855916 17613 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 121 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 121.
I20250111 09:02:03.855966 17336 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 121 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 121.
I20250111 09:02:03.856173 17476 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 122 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:03.856204 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 121 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:02:03.856400 18756 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 121 FOLLOWER]: Leader election won for term 121
I20250111 09:02:03.856457 17476 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 122 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:03.856607 18756 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 121 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:03.856704 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 122 candidate_status { last_received { term: 121 index: 3673 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:03.856719 18756 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3591, Committed index: 3591, Last appended: 120.3591, Last appended by leader: 3591, Current term: 121, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:03.856812 17336 raft_consensus.cc:2388] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 122 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 in current term 122: Already voted for candidate fb270e42d7c9400f8c05bb9565db1972 in this term.
I20250111 09:02:03.857246 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 122 candidate_status { last_received { term: 121 index: 3673 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:03.858044 17613 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 122 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 122.
I20250111 09:02:03.858053 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 120 to 121, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 121 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:03.858264 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 122 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:03.858448 18762 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 122 FOLLOWER]: Leader election won for term 122
I20250111 09:02:03.858506 18762 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 122 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:03.858609 18762 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3671, Committed index: 3671, Last appended: 121.3673, Last appended by leader: 3672, Current term: 122, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:03.860538 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 121 to 122, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 122 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:02:03.862840 18739 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:03.867601 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:03.869984 17613 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 122 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 121 index: 3673. Preceding OpId from leader: term: 122 index: 3675. (index mismatch)
I20250111 09:02:03.870015 17336 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 122 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 121 index: 3670. Preceding OpId from leader: term: 122 index: 3675. (index mismatch)
I20250111 09:02:03.870255 18756 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3674, Last known committed idx: 3671, Time since last communication: 0.000s
I20250111 09:02:03.870365 18762 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3674, Last known committed idx: 3668, Time since last communication: 0.000s
W20250111 09:02:03.883134 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.886539 17300 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.890450 18754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:03.897262 17614 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 121 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 120 index: 3591. Preceding OpId from leader: term: 121 index: 3593. (index mismatch)
I20250111 09:02:03.897564 18756 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3592, Last known committed idx: 3591, Time since last communication: 0.000s
I20250111 09:02:03.898742 17336 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 121 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 120 index: 3591. Preceding OpId from leader: term: 121 index: 3593. (index mismatch)
I20250111 09:02:03.903653 18762 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3592, Last known committed idx: 3591, Time since last communication: 0.000s
W20250111 09:02:03.956246 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.960490 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:03.979092 18614 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 68 similar messages]
W20250111 09:02:04.438755 18591 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 85 similar messages]
I20250111 09:02:04.797744 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:02:04.798243 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:04.800611 17613 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 121 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:04.800695 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 121 FOLLOWER]: Advancing to term 122
I20250111 09:02:04.801769 17613 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 122 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:04.802217 17613 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 122 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:04.802889 17612 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 122 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 121. Current term is 122. Ops: [121.3828-121.3828]
I20250111 09:02:04.803390 18765 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: INVALID_TERM, Last received: 121.3827, Next index: 3828, Last known committed idx: 3826, Time since last communication: 0.000s
I20250111 09:02:04.803452 17612 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:04.804188 18765 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 121 LEADER]: Stepping down as leader of term 121
I20250111 09:02:04.804211 17612 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 122 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:04.804244 18765 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 121 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:04.804275 17612 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 122 FOLLOWER]: Advancing to term 123
I20250111 09:02:04.804356 18765 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3827, Committed index: 3827, Last appended: 121.3832, Last appended by leader: 3832, Current term: 121, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:04.804752 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 122 candidate_status { last_received { term: 121 index: 3827 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:04.804746 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 122 candidate_status { last_received { term: 121 index: 3827 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:04.804838 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 121 FOLLOWER]: Advancing to term 122
I20250111 09:02:04.805138 17612 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 123 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:04.805405 18618 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.001s ago.
I20250111 09:02:04.805464 17612 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 123 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:04.805845 18765 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 121 FOLLOWER]: Advancing to term 122
I20250111 09:02:04.805907 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 123 candidate_status { last_received { term: 122 index: 3894 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:04.805996 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 122 FOLLOWER]: Advancing to term 123
I20250111 09:02:04.806841 17340 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 123 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 123.
I20250111 09:02:04.807001 17336 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 123 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 122. Current term is 123. Ops: [122.3893-122.3894]
I20250111 09:02:04.807452 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 123 candidate_status { last_received { term: 122 index: 3894 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:04.807468 18764 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: INVALID_TERM, Last received: 122.3892, Next index: 3893, Last known committed idx: 3891, Time since last communication: 0.000s
I20250111 09:02:04.807338 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 123 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:04.807667 17476 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 122 LEADER]: Stepping down as leader of term 122
I20250111 09:02:04.807687 18715 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 123 FOLLOWER]: Leader election won for term 123
I20250111 09:02:04.807718 17476 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 122 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:04.807754 18715 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 123 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:04.807837 17476 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3894, Committed index: 3894, Last appended: 122.3895, Last appended by leader: 3895, Current term: 122, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:04.807857 18715 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3893, Committed index: 3893, Last appended: 122.3894, Last appended by leader: 3894, Current term: 123, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:04.808274 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 122 FOLLOWER]: Advancing to term 123
I20250111 09:02:04.809103 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 123 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 123 because replica has last-logged OpId of term: 122 index: 3895, which is greater than that of the candidate, which has last-logged OpId of term: 122 index: 3894.
W20250111 09:02:04.809711 18760 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:04.809809 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 122 to 123, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 123 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:04.810252 17612 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 123 LEADER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 122. Current term is 123. Ops: [122.3895-122.3895]
W20250111 09:02:04.810586 18727 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:04.811728 18771 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 52 similar messages]
W20250111 09:02:04.811815 18771 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:04.812173 17338 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 122 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 122.
I20250111 09:02:04.812394 17335 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 122 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 121. Current term is 122. Ops: [121.3828-121.3828]
I20250111 09:02:04.812515 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 122 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:02:04.812656 18715 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 122 FOLLOWER]: Leader election won for term 122
I20250111 09:02:04.812733 18715 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 122 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:04.812834 18715 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3826, Committed index: 3826, Last appended: 121.3827, Last appended by leader: 3827, Current term: 122, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:04.814010 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 121 to 122, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 122 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
W20250111 09:02:04.814884 18771 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:04.815673 18775 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:04.817816 17474 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 122 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 121 index: 3832. Preceding OpId from leader: term: 122 index: 3829. (term mismatch)
I20250111 09:02:04.817816 17335 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 122 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 121 index: 3827. Preceding OpId from leader: term: 122 index: 3829. (index mismatch)
I20250111 09:02:04.817878 17474 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 3828
I20250111 09:02:04.817901 17474 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 121.3829
W20250111 09:02:04.817991 17474 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
I20250111 09:02:04.818166 18734 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3828, Last known committed idx: 3826, Time since last communication: 0.000s
I20250111 09:02:04.818282 17474 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 121.3830
W20250111 09:02:04.818374 17474 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
I20250111 09:02:04.818533 17474 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 121.3831
W20250111 09:02:04.818650 17474 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
I20250111 09:02:04.818843 17474 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 121.3832
W20250111 09:02:04.818969 17474 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
I20250111 09:02:04.819617 18770 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3828, Last known committed idx: 3827, Time since last communication: 0.000s
I20250111 09:02:04.819712 18770 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer 262623ac6df745c785ce267ea39bd265 log is divergent from this leader: its last log entry 121.3828 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3827
I20250111 09:02:04.820152 17474 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 3827
I20250111 09:02:04.820214 17474 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 121.3828
W20250111 09:02:04.820281 17474 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Aborted: Op aborted by new leader
W20250111 09:02:04.822129 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:04.846165 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:04.846165 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:04.849869 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:04.851789 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:04.852327 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:05.159260 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:02:05.159749 17335 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:05.159857 17335 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 122 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:05.159922 17335 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 122 FOLLOWER]: Advancing to term 123
I20250111 09:02:05.160730 17335 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 123 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:05.161057 17335 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 123 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:05.161314 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 123 candidate_status { last_received { term: 122 index: 3852 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:05.161342 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 123 candidate_status { last_received { term: 122 index: 3852 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:05.161453 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 122 FOLLOWER]: Advancing to term 123
I20250111 09:02:05.161470 17612 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 122 LEADER]: Stepping down as leader of term 122
I20250111 09:02:05.161511 17612 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 122 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:05.161738 17612 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3852, Committed index: 3852, Last appended: 122.3852, Last appended by leader: 3852, Current term: 122, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:05.161893 17612 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 122 FOLLOWER]: Advancing to term 123
I20250111 09:02:05.162309 17476 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 123 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 123.
I20250111 09:02:05.162577 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 123 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:05.162667 17612 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 123 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 123.
I20250111 09:02:05.162734 18777 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 123 FOLLOWER]: Leader election won for term 123
I20250111 09:02:05.162883 18777 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 123 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:05.163020 17335 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:05.163008 18777 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3852, Committed index: 3852, Last appended: 122.3852, Last appended by leader: 3852, Current term: 123, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:05.163182 17335 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 123 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:05.163228 17335 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 123 FOLLOWER]: Advancing to term 124
I20250111 09:02:05.163992 17335 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 124 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:05.164317 17335 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 124 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:05.164570 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 124 candidate_status { last_received { term: 122 index: 3892 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:05.164659 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 123 FOLLOWER]: Advancing to term 124
I20250111 09:02:05.164937 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 124 candidate_status { last_received { term: 122 index: 3892 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:05.165078 17612 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 123 LEADER]: Stepping down as leader of term 123
I20250111 09:02:05.165118 17612 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 123 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:05.165041 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 122 to 123, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 123 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:05.165251 17612 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3893, Committed index: 3893, Last appended: 123.3895, Last appended by leader: 3895, Current term: 123, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:05.165398 17612 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 123 FOLLOWER]: Advancing to term 124
I20250111 09:02:05.165517 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 124 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 124 because replica has last-logged OpId of term: 122 index: 3895, which is greater than that of the candidate, which has last-logged OpId of term: 122 index: 3892.
I20250111 09:02:05.166231 17612 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 124 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 124 because replica has last-logged OpId of term: 123 index: 3895, which is greater than that of the candidate, which has last-logged OpId of term: 122 index: 3892.
I20250111 09:02:05.166575 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 124 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:05.166813 18784 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 124 FOLLOWER]: Leader election lost for term 124. Reason: could not achieve majority
I20250111 09:02:05.278821 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:02:05.279314 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:05.279419 17476 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 123 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:05.279481 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 123 FOLLOWER]: Advancing to term 124
I20250111 09:02:05.280170 17476 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 124 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:05.280489 17476 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 124 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:05.280754 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 124 candidate_status { last_received { term: 122 index: 3852 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:05.280754 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 124 candidate_status { last_received { term: 122 index: 3852 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:05.280864 17612 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 123 FOLLOWER]: Advancing to term 124
I20250111 09:02:05.280864 17335 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 123 LEADER]: Stepping down as leader of term 123
I20250111 09:02:05.280910 17335 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 123 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:05.281067 17335 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3852, Committed index: 3852, Last appended: 123.3853, Last appended by leader: 3853, Current term: 123, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:05.281219 17335 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 123 FOLLOWER]: Advancing to term 124
I20250111 09:02:05.281342 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:05.281436 17476 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 124 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:05.281477 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 124 FOLLOWER]: Advancing to term 125
I20250111 09:02:05.281857 17612 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 124 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 124.
I20250111 09:02:05.282101 17335 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 124 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 124 because replica has last-logged OpId of term: 123 index: 3853, which is greater than that of the candidate, which has last-logged OpId of term: 122 index: 3852.
I20250111 09:02:05.282220 17476 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 125 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:05.282300 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 124 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:02:05.282459 18764 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 124 FOLLOWER]: Leader election won for term 124
I20250111 09:02:05.282522 17476 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 125 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:05.282541 18764 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 124 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:05.282649 18764 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3852, Committed index: 3852, Last appended: 122.3852, Last appended by leader: 3852, Current term: 124, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:05.282791 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 125 candidate_status { last_received { term: 122 index: 3895 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:05.282881 17335 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 124 FOLLOWER]: Advancing to term 125
I20250111 09:02:05.283424 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 125 candidate_status { last_received { term: 122 index: 3895 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:05.283524 17612 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 124 FOLLOWER]: Advancing to term 125
I20250111 09:02:05.283646 17335 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 125 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 125.
I20250111 09:02:05.283869 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 125 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:05.284013 18763 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 125 FOLLOWER]: Leader election won for term 125
I20250111 09:02:05.284091 18763 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 125 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:05.284188 18763 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3894, Committed index: 3894, Last appended: 122.3895, Last appended by leader: 3895, Current term: 125, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:05.284142 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 123 to 124, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 124 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:05.284324 17612 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 125 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 125 because replica has last-logged OpId of term: 123 index: 3895, which is greater than that of the candidate, which has last-logged OpId of term: 122 index: 3895.
I20250111 09:02:05.286840 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 123 to 125, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 125 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:05.673174 17612 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 124 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 122 index: 3852. Preceding OpId from leader: term: 124 index: 3853. (index mismatch)
I20250111 09:02:05.673601 18763 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3853, Last known committed idx: 3852, Time since last communication: 0.000s
I20250111 09:02:05.675745 17335 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 124 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 123 index: 3853. Preceding OpId from leader: term: 124 index: 3853. (term mismatch)
I20250111 09:02:05.675818 17335 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 3852
I20250111 09:02:05.675850 17335 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted NO_OP operation due to leader change: 123.3853
I20250111 09:02:05.675890 17335 raft_consensus.cc:2883] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 124 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:02:05.676230 18763 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3853, Last known committed idx: 3852, Time since last communication: 0.000s
I20250111 09:02:05.697480 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:02:05.698038 17612 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:05.698141 17612 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 124 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:05.698221 17612 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 124 FOLLOWER]: Advancing to term 125
I20250111 09:02:05.698945 17612 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 125 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:05.699244 17612 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 125 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:05.699510 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 125 candidate_status { last_received { term: 124 index: 3853 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:05.699517 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 125 candidate_status { last_received { term: 124 index: 3853 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:05.699620 17335 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 124 FOLLOWER]: Advancing to term 125
I20250111 09:02:05.699622 17476 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 124 LEADER]: Stepping down as leader of term 124
I20250111 09:02:05.699666 17476 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 124 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:05.699762 17612 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:05.699792 17476 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3853, Committed index: 3853, Last appended: 124.3853, Last appended by leader: 3853, Current term: 124, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:05.699849 17612 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 125 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:05.699900 17612 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 125 FOLLOWER]: Advancing to term 126
I20250111 09:02:05.699940 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 124 FOLLOWER]: Advancing to term 125
I20250111 09:02:05.700516 17335 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 125 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 125.
I20250111 09:02:05.700731 17476 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 125 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 125.
I20250111 09:02:05.700707 17612 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 126 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:05.700779 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 125 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:05.700959 18715 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 125 FOLLOWER]: Leader election won for term 125
I20250111 09:02:05.701022 17612 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 126 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:05.701093 18715 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 125 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:05.701247 18715 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3853, Committed index: 3853, Last appended: 124.3853, Last appended by leader: 3853, Current term: 125, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:05.701360 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 126 candidate_status { last_received { term: 123 index: 3895 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:05.701455 17476 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 125 LEADER]: Stepping down as leader of term 125
I20250111 09:02:05.701498 17476 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 125 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:05.701603 17476 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3894, Committed index: 3894, Last appended: 125.3896, Last appended by leader: 3896, Current term: 125, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:05.701704 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 125 FOLLOWER]: Advancing to term 126
I20250111 09:02:05.701968 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 126 candidate_status { last_received { term: 123 index: 3895 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:05.702064 17335 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 125 FOLLOWER]: Advancing to term 126
I20250111 09:02:05.702431 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 126 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 126 because replica has last-logged OpId of term: 125 index: 3896, which is greater than that of the candidate, which has last-logged OpId of term: 123 index: 3895.
I20250111 09:02:05.702862 17335 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 126 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 126.
I20250111 09:02:05.702872 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 124 to 125, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 125 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:05.703117 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 126 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:02:05.703277 18790 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 126 FOLLOWER]: Leader election won for term 126
I20250111 09:02:05.703356 18790 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 126 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:05.703469 18790 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3893, Committed index: 3893, Last appended: 123.3895, Last appended by leader: 3895, Current term: 126, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:05.705457 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 125 to 126, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 126 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:05.917346 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:02:05.917892 17335 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:05.918027 17335 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 125 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:05.918092 17335 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 125 FOLLOWER]: Advancing to term 126
I20250111 09:02:05.919060 17335 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 126 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:05.919405 17335 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 126 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:05.919775 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 126 candidate_status { last_received { term: 124 index: 3853 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:05.919845 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 126 candidate_status { last_received { term: 124 index: 3853 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:05.919930 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 125 FOLLOWER]: Advancing to term 126
I20250111 09:02:05.919981 17612 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 125 LEADER]: Stepping down as leader of term 125
I20250111 09:02:05.920030 17612 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 125 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:05.920200 17612 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3853, Committed index: 3853, Last appended: 125.3854, Last appended by leader: 3854, Current term: 125, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:05.920337 17612 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 125 FOLLOWER]: Advancing to term 126
I20250111 09:02:05.920925 17476 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 126 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 126.
I20250111 09:02:05.921259 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 126 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:05.921289 17335 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:05.921370 17612 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 126 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 126 because replica has last-logged OpId of term: 125 index: 3854, which is greater than that of the candidate, which has last-logged OpId of term: 124 index: 3853.
I20250111 09:02:05.921406 17335 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 126 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:05.921427 18789 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 126 FOLLOWER]: Leader election won for term 126
I20250111 09:02:05.921458 17335 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 126 FOLLOWER]: Advancing to term 127
I20250111 09:02:05.921487 18789 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 126 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:05.921608 18789 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3853, Committed index: 3853, Last appended: 124.3853, Last appended by leader: 3853, Current term: 126, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:05.922057 17335 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 127 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:05.922350 17335 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 127 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:05.922709 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 127 candidate_status { last_received { term: 122 index: 3892 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:05.922763 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 127 candidate_status { last_received { term: 122 index: 3892 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:05.922827 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 126 FOLLOWER]: Advancing to term 127
I20250111 09:02:05.922891 17612 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 126 LEADER]: Stepping down as leader of term 126
I20250111 09:02:05.922937 17612 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 126 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:05.923075 17612 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3893, Committed index: 3893, Last appended: 126.3896, Last appended by leader: 3896, Current term: 126, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:05.923051 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 125 to 126, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 126 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:05.923250 17612 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 126 FOLLOWER]: Advancing to term 127
I20250111 09:02:05.923684 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 127 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 127 because replica has last-logged OpId of term: 125 index: 3896, which is greater than that of the candidate, which has last-logged OpId of term: 122 index: 3892.
I20250111 09:02:05.924052 17612 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 127 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 127 because replica has last-logged OpId of term: 126 index: 3896, which is greater than that of the candidate, which has last-logged OpId of term: 122 index: 3892.
I20250111 09:02:05.924340 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 127 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:05.924490 18787 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 127 FOLLOWER]: Leader election lost for term 127. Reason: could not achieve majority
I20250111 09:02:06.499653 17612 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 126 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 125 index: 3854. Preceding OpId from leader: term: 126 index: 3854. (term mismatch)
I20250111 09:02:06.499722 17612 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 3853
I20250111 09:02:06.499737 17612 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted NO_OP operation due to leader change: 125.3854
I20250111 09:02:06.499756 17612 raft_consensus.cc:2883] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 126 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:02:06.500260 18791 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3854, Last known committed idx: 3853, Time since last communication: 0.000s
I20250111 09:02:06.503273 17476 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 126 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 124 index: 3853. Preceding OpId from leader: term: 126 index: 3854. (index mismatch)
I20250111 09:02:06.503680 18792 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3854, Last known committed idx: 3853, Time since last communication: 0.000s
I20250111 09:02:06.847790 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:02:06.848369 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:06.848489 17476 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 126 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:06.848555 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 126 FOLLOWER]: Advancing to term 127
I20250111 09:02:06.849280 17476 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 127 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:06.849591 17476 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 127 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:06.849897 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 127 candidate_status { last_received { term: 126 index: 3854 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:06.849900 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 127 candidate_status { last_received { term: 126 index: 3854 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:06.850023 17335 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 126 LEADER]: Stepping down as leader of term 126
I20250111 09:02:06.850023 17612 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 126 FOLLOWER]: Advancing to term 127
I20250111 09:02:06.850065 17335 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 126 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:06.850205 17335 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3854, Committed index: 3854, Last appended: 126.3854, Last appended by leader: 3854, Current term: 126, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:06.850347 17335 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 126 FOLLOWER]: Advancing to term 127
I20250111 09:02:06.850845 17612 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 127 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 127.
I20250111 09:02:06.851253 17335 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 127 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 127.
I20250111 09:02:06.851485 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 127 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:02:06.851768 18796 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 127 FOLLOWER]: Leader election won for term 127
I20250111 09:02:06.851770 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:06.851924 17476 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 127 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:06.851977 18796 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 127 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:06.852145 18796 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3854, Committed index: 3854, Last appended: 126.3854, Last appended by leader: 3854, Current term: 127, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:06.851984 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 127 FOLLOWER]: Advancing to term 128
I20250111 09:02:06.853366 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 126 to 127, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 127 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:06.853453 17476 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 128 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:06.853760 17476 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 128 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:06.854147 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 128 candidate_status { last_received { term: 125 index: 3896 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:06.854151 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 128 candidate_status { last_received { term: 125 index: 3896 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:06.854250 17335 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 127 FOLLOWER]: Advancing to term 128
I20250111 09:02:06.854250 17612 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 127 FOLLOWER]: Advancing to term 128
I20250111 09:02:06.855008 17612 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 128 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 128 because replica has last-logged OpId of term: 126 index: 3896, which is greater than that of the candidate, which has last-logged OpId of term: 125 index: 3896.
I20250111 09:02:06.855103 17335 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 128 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 128.
I20250111 09:02:06.855372 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 128 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters: b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:06.855566 18796 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 128 FOLLOWER]: Leader election won for term 128
I20250111 09:02:06.855645 18796 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 128 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:06.855756 18796 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3894, Committed index: 3894, Last appended: 125.3896, Last appended by leader: 3896, Current term: 128, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:06.857344 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 126 to 128, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 128 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:07.094573 18790 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 128 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:02:07.094657 18790 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 128 FOLLOWER]: Advancing to term 129
I20250111 09:02:07.095582 18790 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 129 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:07.095957 18790 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 129 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:07.096223 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 129 candidate_status { last_received { term: 126 index: 3896 } } ignore_live_leader: false dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:07.096232 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 129 candidate_status { last_received { term: 126 index: 3896 } } ignore_live_leader: false dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:07.096334 17335 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 128 FOLLOWER]: Advancing to term 129
I20250111 09:02:07.097296 17335 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 129 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 129.
I20250111 09:02:07.097571 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 129 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:02:07.097745 18790 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 129 FOLLOWER]: Leader election won for term 129
I20250111 09:02:07.097921 18790 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 129 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:07.098071 18790 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3893, Committed index: 3893, Last appended: 126.3896, Last appended by leader: 3896, Current term: 129, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:07.099371 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 128 to 129, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 129 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:07.395607 17612 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 129 LEADER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 128. Current term is 129. Ops: []
I20250111 09:02:07.395977 18805 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3897, Last known committed idx: 3893, Time since last communication: 0.000s
I20250111 09:02:07.396173 18805 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 129.3897 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3893
I20250111 09:02:07.396250 18806 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 128 LEADER]: Stepping down as leader of term 128
I20250111 09:02:07.396311 18806 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 128 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:07.396441 18806 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3894, Committed index: 3894, Last appended: 128.3897, Last appended by leader: 3897, Current term: 128, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:07.396576 18806 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 128 FOLLOWER]: Advancing to term 129
I20250111 09:02:07.410543 17612 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 127 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 126 index: 3854. Preceding OpId from leader: term: 127 index: 3855. (index mismatch)
I20250111 09:02:07.410848 18806 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3855, Last known committed idx: 3854, Time since last communication: 0.000s
I20250111 09:02:07.413168 17335 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 127 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 126 index: 3854. Preceding OpId from leader: term: 127 index: 3855. (index mismatch)
I20250111 09:02:07.413508 18805 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3855, Last known committed idx: 3854, Time since last communication: 0.000s
I20250111 09:02:07.479188 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:02:07.479724 17612 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:07.479820 17612 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 127 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:07.479879 17612 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 127 FOLLOWER]: Advancing to term 128
I20250111 09:02:07.480587 17612 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 128 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:07.480902 17612 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 128 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:07.481210 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 128 candidate_status { last_received { term: 127 index: 3855 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:07.481226 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 128 candidate_status { last_received { term: 127 index: 3855 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:07.481339 17476 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 127 LEADER]: Stepping down as leader of term 127
I20250111 09:02:07.481359 17335 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 127 FOLLOWER]: Advancing to term 128
I20250111 09:02:07.481387 17476 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 127 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:07.481532 17476 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3855, Committed index: 3855, Last appended: 127.3855, Last appended by leader: 3855, Current term: 127, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:07.481678 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 127 FOLLOWER]: Advancing to term 128
I20250111 09:02:07.481972 17612 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:07.482066 17612 raft_consensus.cc:474] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 129 LEADER]: Not starting forced leader election -- already a leader
I20250111 09:02:07.482509 17335 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 128 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 128.
I20250111 09:02:07.482654 17476 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 128 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 128.
I20250111 09:02:07.482839 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 128 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:07.483049 18790 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 128 FOLLOWER]: Leader election won for term 128
I20250111 09:02:07.483124 18790 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 128 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:07.483263 18790 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3855, Committed index: 3855, Last appended: 127.3855, Last appended by leader: 3855, Current term: 128, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:07.484478 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 127 to 128, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 128 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:07.649565 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:02:07.650156 17476 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 129 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 128 index: 3897. Preceding OpId from leader: term: 129 index: 3897. (term mismatch)
I20250111 09:02:07.650218 17476 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 3896
I20250111 09:02:07.650149 17335 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:07.650235 17476 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted NO_OP operation due to leader change: 128.3897
I20250111 09:02:07.650260 17476 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 129 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:02:07.650280 17335 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 128 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:07.650331 17335 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 128 FOLLOWER]: Advancing to term 129
I20250111 09:02:07.650637 18790 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3897, Last known committed idx: 3894, Time since last communication: 0.000s
I20250111 09:02:07.650743 18790 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer 262623ac6df745c785ce267ea39bd265 log is divergent from this leader: its last log entry 125.3896 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3894
I20250111 09:02:07.651229 17476 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 3894
I20250111 09:02:07.651273 17476 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 122.3895
W20250111 09:02:07.651381 17476 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
I20250111 09:02:07.651374 17335 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 129 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:07.651499 17476 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50258 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=4968, attempt_no=0}) took 2849 ms. Trace:
I20250111 09:02:07.651554 17476 rpcz_store.cc:276] 0111 09:02:04.802338 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:04.802536 (+ 198us) service_pool.cc:225] Handling call
0111 09:02:07.651491 (+2848955us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:04.803495 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 6ab63c6543ad4a0ab83e76267a725edd
0111 09:02:04.803546 (+ 51us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:04.803548 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:02:04.803549 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:04.803585 (+ 36us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:02:04.803605 (+ 20us) tablet.cc:698] Row locks acquired
0111 09:02:04.803606 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:02:04.803626 (+ 20us) write_op.cc:270] Start()
0111 09:02:04.803640 (+ 14us) write_op.cc:276] Timestamp: P: 1736586124803584 usec, L: 0
0111 09:02:04.803641 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:02:04.803726 (+ 85us) log.cc:844] Serialized 306 byte log entry
0111 09:02:07.651347 (+2847621us) op_driver.cc:464] REPLICATION: finished
0111 09:02:07.651353 (+ 6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0111 09:02:07.651357 (+ 4us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:07.651370 (+ 13us) write_op.cc:454] Released schema lock
0111 09:02:07.651375 (+ 5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":765,"prepare.run_cpu_time_us":275,"prepare.run_wall_time_us":276,"replication_time_us":2847661}]]}
I20250111 09:02:07.651705 17335 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 129 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:07.651871 17476 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted NO_OP operation due to leader change: 125.3896
I20250111 09:02:07.651921 17476 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 129 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:02:07.652020 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 129 candidate_status { last_received { term: 127 index: 3855 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:07.652135 17612 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 128 LEADER]: Stepping down as leader of term 128
I20250111 09:02:07.652177 17612 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 128 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:07.652320 17612 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3855, Committed index: 3855, Last appended: 128.3856, Last appended by leader: 3856, Current term: 128, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:07.652439 17612 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 128 FOLLOWER]: Advancing to term 129
I20250111 09:02:07.652455 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 129 candidate_status { last_received { term: 127 index: 3855 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:07.652550 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 128 FOLLOWER]: Advancing to term 129
I20250111 09:02:07.653414 17474 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 129 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 129.
I20250111 09:02:07.654116 17612 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 129 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 129 because replica has last-logged OpId of term: 128 index: 3856, which is greater than that of the candidate, which has last-logged OpId of term: 127 index: 3855.
I20250111 09:02:07.654325 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 129 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:07.654551 17335 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:07.654618 18809 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 129 FOLLOWER]: Leader election won for term 129
I20250111 09:02:07.654644 17335 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 129 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:07.654685 17335 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 129 FOLLOWER]: Advancing to term 130
I20250111 09:02:07.654757 18809 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 129 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:07.654872 18809 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3855, Committed index: 3855, Last appended: 127.3855, Last appended by leader: 3855, Current term: 129, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:07.655421 17335 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 130 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:07.655716 17335 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 130 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:07.655992 17338 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 130 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 129. Current term is 130. Ops: []
I20250111 09:02:07.656020 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 130 candidate_status { last_received { term: 122 index: 3892 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:07.656090 17612 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 129 LEADER]: Stepping down as leader of term 129
I20250111 09:02:07.656123 17612 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 129 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:07.656229 17612 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3897, Committed index: 3897, Last appended: 129.3897, Last appended by leader: 3897, Current term: 129, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:07.656322 17612 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 129 FOLLOWER]: Advancing to term 130
I20250111 09:02:07.656340 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 128 to 129, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 129 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:07.656620 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 130 candidate_status { last_received { term: 122 index: 3892 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:07.657135 17612 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 130 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 130 because replica has last-logged OpId of term: 129 index: 3897, which is greater than that of the candidate, which has last-logged OpId of term: 122 index: 3892.
I20250111 09:02:07.657626 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 130 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
W20250111 09:02:07.657781 18814 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 5 similar messages]
W20250111 09:02:07.657871 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:07.658401 18809 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 130 FOLLOWER]: Leader election lost for term 130. Reason: could not achieve majority
W20250111 09:02:07.658816 18600 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:07.658917 18600 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=4965, attempt_no=2}) took 2837 ms. Trace:
I20250111 09:02:07.658954 18600 rpcz_store.cc:276] 0111 09:02:04.821707 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:04.821760 (+ 53us) service_pool.cc:225] Handling call
0111 09:02:07.658909 (+2837149us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:04.827479 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:07.658708 (+2831229us) op_driver.cc:389] HandleFailure(Illegal state: Transaction 637 already open)
0111 09:02:07.658712 (+ 4us) participant_op.cc:94] Released txn lock
0111 09:02:07.658716 (+ 4us) participant_op.cc:287] FINISH: Op aborted
Related trace 'op':
0111 09:02:07.658806 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER)
0111 09:02:07.658809 (+ 3us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:07.658811 (+ 2us) write_op.cc:454] Released schema lock
0111 09:02:07.658813 (+ 2us) write_op.cc:335] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":473,"txn-participant-registration.run_cpu_time_us":373,"txn-participant-registration.run_wall_time_us":5101,"child_traces":[["op",{"prepare.queue_time_us":20,"child_traces":[["op",{}]]}]]}
W20250111 09:02:07.659385 18600 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:07.659494 18600 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=4973, attempt_no=0}) took 2834 ms. Trace:
I20250111 09:02:07.659534 18600 rpcz_store.cc:276] 0111 09:02:04.825412 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:04.825450 (+ 38us) service_pool.cc:225] Handling call
0111 09:02:07.659486 (+2834036us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:07.659252 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:07.659273 (+ 21us) op_driver.cc:389] HandleFailure(Illegal state: Transaction 636 already open)
0111 09:02:07.659279 (+ 6us) participant_op.cc:94] Released txn lock
0111 09:02:07.659284 (+ 5us) participant_op.cc:287] FINISH: Op aborted
Related trace 'op':
0111 09:02:07.659374 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER)
0111 09:02:07.659377 (+ 3us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:07.659379 (+ 2us) write_op.cc:454] Released schema lock
0111 09:02:07.659380 (+ 1us) write_op.cc:335] FINISH: Op aborted
Metrics: {"thread_start_us":89,"threads_started":1,"txn-participant-registration.queue_time_us":1954,"txn-participant-registration.run_cpu_time_us":281,"txn-participant-registration.run_wall_time_us":1457,"child_traces":[["op",{"prepare.queue_time_us":2830257,"child_traces":[["op",{}]]}]]}
W20250111 09:02:07.659931 18600 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.006s ago.
I20250111 09:02:07.660022 18600 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.ParticipateInTransaction from 127.14.61.3:36423 (ReqId={client: c602c61032fa436092728430b0406df9, seq_no=3137, attempt_no=2}) took 2803 ms. Trace:
I20250111 09:02:07.660068 18600 rpcz_store.cc:276] 0111 09:02:04.856227 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:04.856271 (+ 44us) service_pool.cc:225] Handling call
0111 09:02:07.660013 (+2803742us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:07.659884 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:07.659897 (+ 13us) participant_op.cc:199] PREPARE: finished
0111 09:02:07.659917 (+ 20us) op_driver.cc:389] HandleFailure(Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.006s ago.)
0111 09:02:07.659922 (+ 5us) participant_op.cc:94] Released txn lock
0111 09:02:07.659927 (+ 5us) participant_op.cc:287] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"prepare.queue_time_us":2803478}]]}
W20250111 09:02:07.660307 18600 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.006s ago.
I20250111 09:02:07.660385 18600 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.ParticipateInTransaction from 127.14.61.3:36423 (ReqId={client: c602c61032fa436092728430b0406df9, seq_no=3140, attempt_no=1}) took 2801 ms. Trace:
I20250111 09:02:07.660421 18600 rpcz_store.cc:276] 0111 09:02:04.859179 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:04.859207 (+ 28us) service_pool.cc:225] Handling call
0111 09:02:07.660379 (+2801172us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:07.660266 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:07.660280 (+ 14us) participant_op.cc:199] PREPARE: finished
0111 09:02:07.660297 (+ 17us) op_driver.cc:389] HandleFailure(Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.006s ago.)
0111 09:02:07.660300 (+ 3us) participant_op.cc:94] Released txn lock
0111 09:02:07.660303 (+ 3us) participant_op.cc:287] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"prepare.queue_time_us":2800931}]]}
W20250111 09:02:07.660614 18600 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.006s ago.
I20250111 09:02:07.660689 18600 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.ParticipateInTransaction from 127.14.61.3:36423 (ReqId={client: c602c61032fa436092728430b0406df9, seq_no=3135, attempt_no=2}) took 2801 ms. Trace:
I20250111 09:02:07.660717 18600 rpcz_store.cc:276] 0111 09:02:04.859237 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:04.859264 (+ 27us) service_pool.cc:225] Handling call
0111 09:02:07.660684 (+2801420us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:07.660572 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:07.660584 (+ 12us) participant_op.cc:199] PREPARE: finished
0111 09:02:07.660601 (+ 17us) op_driver.cc:389] HandleFailure(Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.006s ago.)
0111 09:02:07.660606 (+ 5us) participant_op.cc:94] Released txn lock
0111 09:02:07.660610 (+ 4us) participant_op.cc:287] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"mutex_wait_us":16,"prepare.queue_time_us":2801223}]]}
W20250111 09:02:07.660921 18600 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.007s ago.
I20250111 09:02:07.661037 18600 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=4981, attempt_no=0}) took 2806 ms. Trace:
I20250111 09:02:07.661078 18600 rpcz_store.cc:276] 0111 09:02:04.854083 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:04.854112 (+ 29us) service_pool.cc:225] Handling call
0111 09:02:07.661030 (+2806918us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:07.660860 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:07.660877 (+ 17us) participant_op.cc:199] PREPARE: finished
0111 09:02:07.660898 (+ 21us) op_driver.cc:389] HandleFailure(Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.007s ago.)
0111 09:02:07.660902 (+ 4us) participant_op.cc:94] Released txn lock
0111 09:02:07.660910 (+ 8us) participant_op.cc:287] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":46,"txn-participant-registration.run_cpu_time_us":293,"txn-participant-registration.run_wall_time_us":10294,"child_traces":[["op",{"prepare.queue_time_us":2796282}]]}
W20250111 09:02:07.661370 18600 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.007s ago.
I20250111 09:02:07.661482 18600 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=4982, attempt_no=0}) took 2805 ms. Trace:
I20250111 09:02:07.661522 18600 rpcz_store.cc:276] 0111 09:02:04.855621 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:04.855656 (+ 35us) service_pool.cc:225] Handling call
0111 09:02:07.661475 (+2805819us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:07.661315 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:07.661331 (+ 16us) participant_op.cc:199] PREPARE: finished
0111 09:02:07.661347 (+ 16us) op_driver.cc:389] HandleFailure(Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.007s ago.)
0111 09:02:07.661351 (+ 4us) participant_op.cc:94] Released txn lock
0111 09:02:07.661357 (+ 6us) participant_op.cc:287] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":3117,"txn-participant-registration.run_cpu_time_us":274,"txn-participant-registration.run_wall_time_us":12698,"child_traces":[["op",{"prepare.queue_time_us":2790600}]]}
W20250111 09:02:07.661818 18600 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.008s ago.
I20250111 09:02:07.661921 18600 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=4984, attempt_no=0}) took 2801 ms. Trace:
I20250111 09:02:07.661960 18600 rpcz_store.cc:276] 0111 09:02:04.860602 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:04.860647 (+ 45us) service_pool.cc:225] Handling call
0111 09:02:07.661912 (+2801265us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:07.661760 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:07.661776 (+ 16us) participant_op.cc:199] PREPARE: finished
0111 09:02:07.661794 (+ 18us) op_driver.cc:389] HandleFailure(Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.008s ago.)
0111 09:02:07.661800 (+ 6us) participant_op.cc:94] Released txn lock
0111 09:02:07.661806 (+ 6us) participant_op.cc:287] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":3822,"txn-participant-registration.run_cpu_time_us":281,"txn-participant-registration.run_wall_time_us":6371,"child_traces":[["op",{"prepare.queue_time_us":2790788}]]}
W20250111 09:02:07.662218 18600 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.008s ago.
I20250111 09:02:07.662307 18600 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=4986, attempt_no=0}) took 2783 ms. Trace:
I20250111 09:02:07.662345 18600 rpcz_store.cc:276] 0111 09:02:04.878939 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:04.878971 (+ 32us) service_pool.cc:225] Handling call
0111 09:02:07.662300 (+2783329us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:07.662174 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:07.662190 (+ 16us) participant_op.cc:199] PREPARE: finished
0111 09:02:07.662203 (+ 13us) op_driver.cc:389] HandleFailure(Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.008s ago.)
0111 09:02:07.662206 (+ 3us) participant_op.cc:94] Released txn lock
0111 09:02:07.662210 (+ 4us) participant_op.cc:287] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":34,"txn-participant-registration.run_cpu_time_us":302,"txn-participant-registration.run_wall_time_us":4940,"child_traces":[["op",{"prepare.queue_time_us":2778076}]]}
W20250111 09:02:07.662614 18600 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.008s ago.
I20250111 09:02:07.662710 18600 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=4988, attempt_no=0}) took 2776 ms. Trace:
I20250111 09:02:07.662748 18600 rpcz_store.cc:276] 0111 09:02:04.886476 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:04.886511 (+ 35us) service_pool.cc:225] Handling call
0111 09:02:07.662704 (+2776193us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:07.662565 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:07.662581 (+ 16us) participant_op.cc:199] PREPARE: finished
0111 09:02:07.662596 (+ 15us) op_driver.cc:389] HandleFailure(Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.008s ago.)
0111 09:02:07.662600 (+ 4us) participant_op.cc:94] Released txn lock
0111 09:02:07.662605 (+ 5us) participant_op.cc:287] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":38,"txn-participant-registration.run_cpu_time_us":338,"txn-participant-registration.run_wall_time_us":3381,"child_traces":[["op",{"prepare.queue_time_us":2772518}]]}
W20250111 09:02:07.664407 18816 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 2 similar messages]
W20250111 09:02:07.664454 18816 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.665421 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.666743 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.667048 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.668324 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.668867 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.669052 18818 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.669062 18816 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.669482 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.669559 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.669579 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.669986 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.673661 18822 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 29 similar messages]
W20250111 09:02:07.673723 18822 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.675130 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.675467 18822 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.676024 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.676151 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.676537 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.677544 18822 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.678057 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.678946 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.679098 18822 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.679498 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.681514 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.682094 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.683283 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.683939 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.685380 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.686439 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.686908 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.687292 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.687973 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.688127 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.688558 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.689355 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.690012 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.690292 18822 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.692605 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.694202 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.695430 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.695881 18822 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.696199 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.697853 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.698019 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.698348 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.698840 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.698937 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.701027 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.702375 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.704511 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.705910 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.706910 18822 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.707142 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.707418 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.708839 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.709118 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.710614 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.710675 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.711499 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.713614 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.713680 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.715157 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.715691 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.718616 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.718663 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.718789 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.719497 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.720721 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.721738 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.723500 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.724457 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.724959 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.725581 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.728502 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.729931 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.730245 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.732230 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.732523 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.732685 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.736693 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.738407 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.739015 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.739118 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.739614 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.740684 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.740813 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.746645 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.746744 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.747233 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.747812 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.749790 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.751612 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.751893 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.752547 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.753232 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.754176 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.756089 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.759439 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.759963 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.760756 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.760775 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.763559 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.764883 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.766642 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.767117 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.767755 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.768265 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.772034 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.773113 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.775763 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.775830 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.777737 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.781288 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.781324 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.782639 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.783953 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.784224 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.785032 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.786792 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.788973 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.790798 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.792208 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.793175 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.797443 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.798864 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.798979 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.800884 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.801223 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.801738 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.803289 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.807297 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.808539 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.808832 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.811753 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.814786 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.815562 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.818236 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.818876 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.818929 18822 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.819419 18825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.822006 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.825646 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.826293 18825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.828052 18825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.832257 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.834471 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.835976 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.837342 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.837843 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.841482 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.841646 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.841853 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.844197 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.846947 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.847072 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.851962 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.854916 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.858618 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.858655 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.859736 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.860177 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.860199 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.861558 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.862339 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.864957 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.868052 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.874085 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.874085 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.878027 18825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.878057 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.879315 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.880151 18825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.881515 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.881821 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.883889 18825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.884260 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.888005 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.896739 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.896984 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.897107 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.898092 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.899920 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.903174 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.903754 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.904549 17591 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.906638 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.908623 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.908792 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.918582 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.918838 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.919821 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.920111 18826 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.923642 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.924468 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.925081 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.929132 18826 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.929268 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.930222 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.933847 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.939705 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.941123 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.943871 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.944751 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.947463 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.948993 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.951918 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.952168 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.952276 18825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.955121 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.959003 18825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.966809 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.966933 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:07.967731 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:07.970747 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:07.971030 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:02:07.971485 17474 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:07.971601 17474 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 129 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:07.971665 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 129 FOLLOWER]: Advancing to term 130
W20250111 09:02:07.971693 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:07.972465 17474 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 130 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:07.972756 17474 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 130 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:07.972993 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 130 candidate_status { last_received { term: 127 index: 3855 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:07.972992 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 130 candidate_status { last_received { term: 127 index: 3855 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:07.973114 17612 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 129 FOLLOWER]: Advancing to term 130
I20250111 09:02:07.973114 17335 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 129 LEADER]: Stepping down as leader of term 129
I20250111 09:02:07.973150 17335 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 129 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
W20250111 09:02:07.973289 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:07.973281 17335 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3855, Committed index: 3855, Last appended: 129.3856, Last appended by leader: 3856, Current term: 129, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:07.973397 17335 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 129 FOLLOWER]: Advancing to term 130
I20250111 09:02:07.974071 17612 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 130 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 130 because replica has last-logged OpId of term: 128 index: 3856, which is greater than that of the candidate, which has last-logged OpId of term: 127 index: 3855.
I20250111 09:02:07.974355 17335 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 130 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 130 because replica has last-logged OpId of term: 129 index: 3856, which is greater than that of the candidate, which has last-logged OpId of term: 127 index: 3855.
I20250111 09:02:07.974632 17474 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:07.974668 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 130 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:07.974730 17474 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 129 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:07.974789 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 129 FOLLOWER]: Advancing to term 130
I20250111 09:02:07.974900 18815 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 130 FOLLOWER]: Leader election lost for term 130. Reason: could not achieve majority
I20250111 09:02:07.975483 17474 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 130 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:07.975812 17474 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 130 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
W20250111 09:02:07.976128 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:07.976150 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 130 candidate_status { last_received { term: 129 index: 3897 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:07.976150 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 130 candidate_status { last_received { term: 129 index: 3897 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:07.976293 17335 raft_consensus.cc:2388] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 130 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 in current term 130: Already voted for candidate fb270e42d7c9400f8c05bb9565db1972 in this term.
W20250111 09:02:07.976881 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:07.977087 17612 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 130 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 130.
I20250111 09:02:07.977414 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 130 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:07.977592 18815 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 130 FOLLOWER]: Leader election won for term 130
I20250111 09:02:07.977690 18815 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 130 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:07.977828 18815 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3897, Committed index: 3897, Last appended: 129.3897, Last appended by leader: 3897, Current term: 130, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:07.978785 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:07.978982 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 129 to 130, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 130 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:02:07.981698 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:07.986758 17335 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 130 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 122 index: 3892. Preceding OpId from leader: term: 130 index: 3899. (index mismatch)
I20250111 09:02:07.986771 17612 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 130 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 129 index: 3897. Preceding OpId from leader: term: 130 index: 3899. (index mismatch)
I20250111 09:02:07.987079 18805 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3898, Last known committed idx: 3897, Time since last communication: 0.000s
I20250111 09:02:07.987185 18815 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3898, Last known committed idx: 3891, Time since last communication: 0.000s
W20250111 09:02:07.997663 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.008899 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.018981 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.020144 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.020496 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.022943 18825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.023116 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.024474 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.025525 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.026029 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.030618 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.032438 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.032725 17433 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.033969 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.034297 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.034642 18825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.040241 18820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.040494 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.040912 18817 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.041903 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.041934 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.043732 18825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.045308 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.046985 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.047070 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.047617 18826 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.049325 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.049369 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.050542 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.051218 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.053392 18825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.053547 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.053556 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.054435 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.056193 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.056737 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.056754 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.058645 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.060217 18825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.062927 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.063275 18825 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.063745 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.064550 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.064625 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.067283 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.070205 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.070636 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.072165 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.072851 18826 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.073016 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.074209 18826 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.077522 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.078226 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.079857 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.080117 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.081614 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.083866 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.085813 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.087440 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.089363 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.090077 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.090777 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.091743 18826 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.092468 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.095335 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.095356 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.097101 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.100978 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.101629 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.103317 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.107383 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.107638 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.109109 18826 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.109365 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.109661 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.111819 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.117568 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.117698 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.120675 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.121592 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.121642 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.123428 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.123482 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.127071 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.129098 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.129849 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.134738 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.136198 18824 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.136585 18826 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.137102 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.137825 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.141592 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.143360 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.145231 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.149538 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.150355 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.151183 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.152586 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.154570 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.155062 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.160647 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.160840 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.165568 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.168292 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.168437 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.169214 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.170766 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.171416 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.175472 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.176908 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.183794 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.184973 18826 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.185093 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.185918 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.187351 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.189761 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.192641 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.192662 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.200827 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.202328 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.202463 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.204131 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.204895 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.208739 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.212728 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.214740 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.220402 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.222899 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.223079 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.224356 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.226248 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.230257 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.232476 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.233237 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.241082 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.243381 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.244462 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.244987 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.247850 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.253476 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.254207 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.254329 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.262462 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.262784 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.263312 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.267992 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.269896 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.275460 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.277156 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.277247 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.283077 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.283849 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.284466 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.290381 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.292248 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.298462 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.300390 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.300810 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.305922 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.308881 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.310056 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.310374 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.315353 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.323356 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.324074 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.324767 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.330561 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.333300 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.334900 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.335547 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.340644 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.347811 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.349047 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.352079 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.357270 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.361917 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.363065 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.363513 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.364881 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.373085 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.375358 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.378424 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.381549 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.387537 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.388224 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.388988 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.390836 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.400738 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.402557 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.405555 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.406625 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.416318 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.417846 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.418903 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.421397 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.428992 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.430032 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.434912 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.436429 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.444527 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.444895 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.446530 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.451260 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.457422 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.458508 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.463459 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.464834 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.471772 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.474414 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.474969 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.479732 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.485723 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.486604 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.493075 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.493592 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.500536 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.503976 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.507293 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.509358 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.517077 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.518326 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.523488 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.524789 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.533053 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.534539 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.538929 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.541898 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.548966 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.552268 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.555557 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.556422 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.564132 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.566996 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.573431 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.576418 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.582224 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.586869 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.590318 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.590854 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.597795 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.599581 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.604573 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.613198 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.615061 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.619453 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.624694 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.627050 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.632287 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.636862 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.640344 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.649266 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.650369 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.651510 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.660980 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.661775 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.668052 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.670940 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.674145 18814 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 72 similar messages]
W20250111 09:02:08.674199 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.685165 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.685165 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.685947 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.695035 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.695781 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.703742 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.708353 18821 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 142 similar messages]
W20250111 09:02:08.708431 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.710587 18823 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 80 similar messages]
W20250111 09:02:08.710636 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.721403 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.722934 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.722963 18819 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.731434 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.735613 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.740916 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.747201 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.748749 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.758153 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.758217 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.762388 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.771301 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.772594 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.777732 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.785900 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.786427 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.794256 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.798900 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.801652 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.810920 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.811741 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.815519 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.827294 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.827486 18814 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:08.833962 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:08.836208 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:02:08.836622 17612 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:08.836721 17612 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 130 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:08.836779 17612 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 130 FOLLOWER]: Advancing to term 131
I20250111 09:02:08.837870 17612 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 131 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:08.838220 17612 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 131 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:08.838616 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 131 candidate_status { last_received { term: 128 index: 3856 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:08.838721 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 130 FOLLOWER]: Advancing to term 131
I20250111 09:02:08.838711 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 131 candidate_status { last_received { term: 128 index: 3856 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:08.838824 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 130 FOLLOWER]: Advancing to term 131
I20250111 09:02:08.839651 17474 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 131 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 131.
W20250111 09:02:08.839735 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:08.839812 17338 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 131 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 131 because replica has last-logged OpId of term: 129 index: 3856, which is greater than that of the candidate, which has last-logged OpId of term: 128 index: 3856.
I20250111 09:02:08.839989 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 131 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:02:08.840133 18844 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 131 FOLLOWER]: Leader election won for term 131
I20250111 09:02:08.840278 18844 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 131 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:08.840320 17612 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:08.840389 17612 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 130 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:08.840430 17612 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 130 FOLLOWER]: Advancing to term 131
I20250111 09:02:08.840409 18844 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3855, Committed index: 3855, Last appended: 128.3856, Last appended by leader: 3856, Current term: 131, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:08.841158 17612 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 131 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:08.841465 17612 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 131 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:08.841689 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 131 candidate_status { last_received { term: 130 index: 3923 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:08.841789 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 130 FOLLOWER]: Advancing to term 131
I20250111 09:02:08.841982 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 129 to 131, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 131 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:08.842108 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 131 candidate_status { last_received { term: 130 index: 3923 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:08.842185 17474 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 130 LEADER]: Stepping down as leader of term 130
I20250111 09:02:08.842218 17474 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 130 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:08.842331 17474 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3923, Committed index: 3923, Last appended: 130.3923, Last appended by leader: 3923, Current term: 130, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:08.842485 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 130 FOLLOWER]: Advancing to term 131
I20250111 09:02:08.842684 17338 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 131 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 131.
I20250111 09:02:08.842955 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 131 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:08.843102 18848 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 131 FOLLOWER]: Leader election won for term 131
I20250111 09:02:08.843214 18848 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 131 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:08.843305 17474 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 131 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 131.
I20250111 09:02:08.843334 18848 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3923, Committed index: 3923, Last appended: 130.3923, Last appended by leader: 3923, Current term: 131, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:08.845153 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:08.846619 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 130 to 131, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 131 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:08.850864 17474 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 131 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 127 index: 3855. Preceding OpId from leader: term: 131 index: 3858. (index mismatch)
I20250111 09:02:08.850874 17338 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 131 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 129 index: 3856. Preceding OpId from leader: term: 131 index: 3858. (index mismatch)
I20250111 09:02:08.851191 18848 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3857, Last known committed idx: 3855, Time since last communication: 0.000s
I20250111 09:02:08.851271 18848 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer fb270e42d7c9400f8c05bb9565db1972 log is divergent from this leader: its last log entry 129.3856 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3855
I20250111 09:02:08.851327 18844 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3857, Last known committed idx: 3855, Time since last communication: 0.000s
I20250111 09:02:08.851881 17338 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 3855
I20250111 09:02:08.851922 17338 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted NO_OP operation due to leader change: 129.3856
I20250111 09:02:08.851941 17338 raft_consensus.cc:2883] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 131 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250111 09:02:08.856498 18855 mvcc.cc:204] Tried to move back new op lower bound from 7113056783770439680 to 7113056783731605504. Current Snapshot: MvccSnapshot[applied={T|T < 7113056783770439680}]
I20250111 09:02:08.857714 18856 mvcc.cc:204] Tried to move back new op lower bound from 7113056783770439680 to 7113056783731605504. Current Snapshot: MvccSnapshot[applied={T|T < 7113056783770439680 or (T in {7113056783770439680})}]
I20250111 09:02:08.862125 17474 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 131 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 130 index: 3923. Preceding OpId from leader: term: 131 index: 3925. (index mismatch)
I20250111 09:02:08.862397 18844 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3924, Last known committed idx: 3923, Time since last communication: 0.000s
I20250111 09:02:08.862537 17338 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 131 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 130 index: 3923. Preceding OpId from leader: term: 131 index: 3925. (index mismatch)
I20250111 09:02:08.862825 18848 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3924, Last known committed idx: 3923, Time since last communication: 0.000s
W20250111 09:02:08.884464 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.891409 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:08.891657 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:09.166464 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:02:09.166978 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:09.167552 17340 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 131 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:09.167624 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 131 FOLLOWER]: Advancing to term 132
I20250111 09:02:09.168474 17340 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 132 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:09.168785 17340 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 132 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:09.169190 17473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 132 candidate_status { last_received { term: 131 index: 3944 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:09.169291 17473 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 131 FOLLOWER]: Advancing to term 132
I20250111 09:02:09.169978 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:09.170070 17340 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 131 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:09.170120 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 131 FOLLOWER]: Advancing to term 132
I20250111 09:02:09.170990 17340 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 132 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:09.171026 17473 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 132 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 132 because replica has last-logged OpId of term: 131 index: 3946, which is greater than that of the candidate, which has last-logged OpId of term: 131 index: 3944.
I20250111 09:02:09.171324 17340 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 132 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:09.171696 17473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 132 candidate_status { last_received { term: 131 index: 3990 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:09.173199 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 132 candidate_status { last_received { term: 131 index: 3944 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:09.173271 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 132 candidate_status { last_received { term: 131 index: 3990 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:09.173380 17611 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 131 LEADER]: Stepping down as leader of term 131
I20250111 09:02:09.173424 17611 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 131 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:09.173309 17612 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 131 LEADER]: Stepping down as leader of term 131
I20250111 09:02:09.173555 17612 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 131 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:09.173691 17612 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3946, Committed index: 3946, Last appended: 131.3950, Last appended by leader: 3950, Current term: 131, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:09.173791 17612 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 131 FOLLOWER]: Advancing to term 132
I20250111 09:02:09.173556 17611 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3990, Committed index: 3990, Last appended: 131.3994, Last appended by leader: 3994, Current term: 131, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:09.174034 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 131 FOLLOWER]: Advancing to term 132
I20250111 09:02:09.174168 17336 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 132 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 131. Current term is 132. Ops: [131.3991-131.3993]
I20250111 09:02:09.174686 17612 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 132 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 132 because replica has last-logged OpId of term: 131 index: 3950, which is greater than that of the candidate, which has last-logged OpId of term: 131 index: 3944.
I20250111 09:02:09.174893 17611 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 132 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 132 because replica has last-logged OpId of term: 131 index: 3994, which is greater than that of the candidate, which has last-logged OpId of term: 131 index: 3990.
I20250111 09:02:09.174883 17336 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 132 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 131. Current term is 132. Ops: [131.3945-131.3950]
I20250111 09:02:09.175405 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 132 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:09.175565 18860 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 132 FOLLOWER]: Leader election lost for term 132. Reason: could not achieve majority
I20250111 09:02:09.175634 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 132 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:09.175751 18860 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 132 FOLLOWER]: Leader election lost for term 132. Reason: could not achieve majority
I20250111 09:02:09.175973 17476 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 132 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 131. Current term is 132. Ops: [131.3947-131.3949]
W20250111 09:02:09.179292 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.179379 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.182585 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.182585 18890 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.183902 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.185446 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.185886 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.187230 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.188692 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.188966 18865 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.189751 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.191931 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.192971 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.193126 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.194586 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.195116 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.195308 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.197140 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.200474 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.200903 18865 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.202934 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.202976 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.208961 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.209558 18823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.210318 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.210783 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.210778 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.212926 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.215842 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.218115 18865 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.218706 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.219156 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.219467 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.224442 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.226357 18821 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.226693 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.228292 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.230098 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.231478 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.236541 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.236670 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.237936 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.238102 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.241429 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.244649 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.246743 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.247603 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.250010 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.250147 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.254101 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.257210 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.258713 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.259524 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.261545 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.262881 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.264901 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.268545 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.270579 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.272017 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.272888 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.276669 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.279824 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.281526 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.284283 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.284860 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.285950 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.293788 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.293910 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.295313 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.298795 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.300652 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.301393 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.309127 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.311439 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.312808 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.315750 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.318974 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.319247 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.326448 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.326560 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.328943 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.334462 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.335358 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.337042 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.342335 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.343631 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.349977 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.352238 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.354095 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.355509 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.365105 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.366969 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.368441 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.372686 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.375120 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.375944 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.385643 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.386626 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.387547 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.394685 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.396174 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.397539 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.406652 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.407851 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.410233 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.417807 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.418190 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.418900 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.429162 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.429162 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.432579 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.440382 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.440742 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.442828 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.452220 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.452697 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.453701 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.463191 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.463860 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.468020 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.474987 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.478864 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.480295 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.485880 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.486871 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.491824 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.497880 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.502360 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.506775 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.509716 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.509914 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.517534 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.524135 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.526892 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.532718 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.534695 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.535825 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.542325 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.548677 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.552327 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.560119 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.562534 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.566115 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.570359 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.578372 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.580524 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.587273 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.591933 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.592013 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.597740 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.607143 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.609850 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.618938 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.620764 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.620829 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.627764 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.634752 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.642426 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.649091 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.653280 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.653448 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.658119 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.664151 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.673759 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.681454 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.682166 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.684351 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.689854 18891 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 70 similar messages]
W20250111 09:02:09.689899 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.696123 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.705991 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.712239 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.712808 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:09.713439 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:02:09.713829 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:09.713920 17476 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 132 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:09.713979 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 132 FOLLOWER]: Advancing to term 133
I20250111 09:02:09.714694 17476 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 133 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:09.715035 17476 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 133 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:09.715281 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 133 candidate_status { last_received { term: 131 index: 3946 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:09.715330 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 133 candidate_status { last_received { term: 131 index: 3946 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:09.715392 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 132 FOLLOWER]: Advancing to term 133
I20250111 09:02:09.715477 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 132 FOLLOWER]: Advancing to term 133
I20250111 09:02:09.716120 17336 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 133 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 133.
I20250111 09:02:09.716188 17611 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 133 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 133 because replica has last-logged OpId of term: 131 index: 3950, which is greater than that of the candidate, which has last-logged OpId of term: 131 index: 3946.
I20250111 09:02:09.716403 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 133 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:09.716436 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:09.716508 17476 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 131 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:09.716565 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 131 FOLLOWER]: Advancing to term 132
I20250111 09:02:09.716699 18895 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 133 FOLLOWER]: Leader election won for term 133
I20250111 09:02:09.716822 18895 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 133 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:09.716945 18895 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3944, Committed index: 3944, Last appended: 131.3946, Last appended by leader: 3946, Current term: 133, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:09.717361 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Service unavailable: leader is not yet ready
I20250111 09:02:09.717456 17476 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 132 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:09.717765 17476 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 132 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:09.718127 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 132 candidate_status { last_received { term: 131 index: 3992 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:09.718127 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 132 candidate_status { last_received { term: 131 index: 3992 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:09.718263 17611 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 132 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 132 because replica has last-logged OpId of term: 131 index: 3994, which is greater than that of the candidate, which has last-logged OpId of term: 131 index: 3992.
I20250111 09:02:09.718267 17336 raft_consensus.cc:2388] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 132 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 in current term 132: Already voted for candidate fb270e42d7c9400f8c05bb9565db1972 in this term.
I20250111 09:02:09.718722 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 132 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:09.718912 18895 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 132 FOLLOWER]: Leader election lost for term 132. Reason: could not achieve majority
I20250111 09:02:09.719331 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 131 to 133, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 133 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:02:09.721536 18894 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 48 similar messages]
W20250111 09:02:09.721581 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.730612 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:09.741220 17611 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 133 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 131 index: 3950. Preceding OpId from leader: term: 133 index: 3948. (term mismatch)
I20250111 09:02:09.741274 17611 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 3947
I20250111 09:02:09.741298 17611 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 131.3948
I20250111 09:02:09.741343 17336 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 133 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 131 index: 3944. Preceding OpId from leader: term: 133 index: 3948. (index mismatch)
W20250111 09:02:09.741371 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Aborted: Op aborted by new leader
I20250111 09:02:09.741524 17611 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 131.3949
W20250111 09:02:09.741619 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:02:09.741647 18896 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3947, Last known committed idx: 3943, Time since last communication: 0.000s
I20250111 09:02:09.741797 17611 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 131.3950
W20250111 09:02:09.741886 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:02:09.742378 18896 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3947, Last known committed idx: 3946, Time since last communication: 0.000s
I20250111 09:02:09.742470 18896 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 131.3947 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3946
I20250111 09:02:09.742884 17611 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 3946
I20250111 09:02:09.742924 17611 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 131.3947
W20250111 09:02:09.742969 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Aborted: Op aborted by new leader
W20250111 09:02:09.747077 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.748515 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.760252 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.762915 18911 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 52 similar messages]
W20250111 09:02:09.762964 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.763688 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.770239 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.773020 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.779021 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.780184 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.780304 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.780408 18919 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.783087 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.783497 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.785195 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.788118 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.789130 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.789683 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.792850 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.794106 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.795426 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.797667 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.797924 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.798470 18919 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.802052 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.804003 18919 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.805307 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.806143 18919 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.810973 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.813393 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.814181 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.814829 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.815161 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.820601 18919 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.821453 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.825127 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.825184 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.827968 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.833526 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.835638 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.836458 18919 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.837878 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.840015 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.845685 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.848335 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.849556 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.852540 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.853730 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.856233 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.858911 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.860652 18919 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.862111 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.867251 18919 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.871011 18919 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.874256 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.875703 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.880688 18919 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.881071 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.884219 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.888094 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.890409 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.890470 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.895588 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.896960 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.898085 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.906877 18919 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.912600 18919 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.912612 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.913547 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.915746 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.926019 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.926726 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.927191 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.928365 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.932241 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.937117 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.944546 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.946856 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.948002 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.949437 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.959106 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.963780 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.964414 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.966200 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.968009 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.969897 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:09.978865 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.986263 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.986423 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.990391 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:09.991900 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.003082 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.003571 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.005344 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.006404 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.006413 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.015323 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.028534 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.028610 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.029636 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.030526 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.035823 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.042841 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.048327 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.051208 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.051280 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.054965 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.059805 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.069505 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.073094 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.077363 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.079385 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.082952 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.085275 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.091531 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.096868 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.101826 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.103138 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.107836 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.110096 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.122666 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.127681 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.128715 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.131422 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.134967 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.136222 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.149852 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.152725 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.156271 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.159247 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.160548 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.170895 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.178232 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.180145 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.185068 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.187696 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.189639 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.197202 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.210532 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.212646 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.216205 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.216382 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.219031 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.222774 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.243233 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.244367 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.248940 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.249217 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.249625 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.260360 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.267927 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.273958 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.281912 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.282372 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.283195 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.289754 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.305667 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.307617 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.312390 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.313483 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.314796 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.316183 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.334944 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.338137 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.346504 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.348928 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.349090 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.355777 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.362264 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.371506 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.381330 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.382768 18891 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.383381 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.383401 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.404297 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.405552 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.411973 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.416354 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.417377 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.417472 18923 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.433573 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:10.439363 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:02:10.439786 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:10.439884 17611 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 133 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:10.439949 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 133 FOLLOWER]: Advancing to term 134
I20250111 09:02:10.440745 17611 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 134 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:10.441063 17611 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 134 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:10.441246 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 134 candidate_status { last_received { term: 133 index: 3963 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
W20250111 09:02:10.441319 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:10.441356 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 133 FOLLOWER]: Advancing to term 134
I20250111 09:02:10.441576 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 134 candidate_status { last_received { term: 133 index: 3963 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:10.441620 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:10.441677 17476 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 133 LEADER]: Stepping down as leader of term 133
I20250111 09:02:10.441715 17476 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 133 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:10.441715 17611 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 132 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:10.441767 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 132 FOLLOWER]: Advancing to term 133
I20250111 09:02:10.441840 17476 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3963, Committed index: 3963, Last appended: 133.3963, Last appended by leader: 3963, Current term: 133, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:10.441982 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 133 FOLLOWER]: Advancing to term 134
I20250111 09:02:10.442324 17336 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 134 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 134.
I20250111 09:02:10.442464 17611 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 133 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:10.442622 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 134 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:10.442745 17611 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 133 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:10.442930 17476 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 134 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 134.
I20250111 09:02:10.443006 18922 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 134 FOLLOWER]: Leader election won for term 134
I20250111 09:02:10.443055 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 133 candidate_status { last_received { term: 131 index: 3994 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:10.443151 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 132 FOLLOWER]: Advancing to term 133
I20250111 09:02:10.443166 18922 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 134 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:10.443277 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 133 candidate_status { last_received { term: 131 index: 3994 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:10.443362 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 132 FOLLOWER]: Advancing to term 133
I20250111 09:02:10.443337 18922 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3963, Committed index: 3963, Last appended: 133.3963, Last appended by leader: 3963, Current term: 134, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:10.443889 17336 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 133 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 133.
I20250111 09:02:10.444192 17467 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 133 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 133.
I20250111 09:02:10.444233 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 133 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:10.444365 18924 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 133 FOLLOWER]: Leader election won for term 133
I20250111 09:02:10.444476 18924 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 133 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:10.444604 18924 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3990, Committed index: 3990, Last appended: 131.3994, Last appended by leader: 3994, Current term: 133, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:10.444702 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 133 to 134, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 134 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:10.447640 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 131 to 133. New cstate: current_term: 133 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
W20250111 09:02:10.450301 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:10.453167 17336 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 133 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 131 index: 3990. Preceding OpId from leader: term: 133 index: 3996. (index mismatch)
I20250111 09:02:10.453189 17467 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 133 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 131 index: 3992. Preceding OpId from leader: term: 133 index: 3996. (index mismatch)
I20250111 09:02:10.453523 18922 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3995, Last known committed idx: 3990, Time since last communication: 0.000s
I20250111 09:02:10.453703 18924 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3995, Last known committed idx: 3990, Time since last communication: 0.000s
W20250111 09:02:10.455269 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:10.457326 18937 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.ParticipateInTransaction from 127.14.61.3:36423 (ReqId={client: c602c61032fa436092728430b0406df9, seq_no=3283, attempt_no=0}) took 1292 ms. Trace:
I20250111 09:02:10.457818 18929 mvcc.cc:204] Tried to move back new op lower bound from 7113056790333419520 to 7113056790302318592. Current Snapshot: MvccSnapshot[applied={T|T < 7113056784899260416 or (T in {7113056784941064192,7113056784953200640,7113056785015803904})}]
I20250111 09:02:10.457455 18937 rpcz_store.cc:276] 0111 09:02:09.164667 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:09.164774 (+ 107us) service_pool.cc:225] Handling call
0111 09:02:10.457312 (+1292538us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:09.166518 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:09.166545 (+ 27us) participant_op.cc:199] PREPARE: finished
0111 09:02:09.166559 (+ 14us) participant_op.cc:212] START. Timestamp: 1736586129166486
0111 09:02:09.166561 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:02:09.166622 (+ 61us) log.cc:844] Serialized 68 byte log entry
0111 09:02:10.456567 (+1289945us) op_driver.cc:464] REPLICATION: finished
0111 09:02:10.456849 (+ 282us) participant_op.cc:265] APPLY: starting
0111 09:02:10.456949 (+ 100us) participant_op.cc:270] APPLY: finished
0111 09:02:10.457256 (+ 307us) log.cc:844] Serialized 16 byte log entry
0111 09:02:10.457285 (+ 29us) participant_op.cc:94] Released txn lock
0111 09:02:10.457298 (+ 13us) participant_op.cc:291] FINISH: Op applied
Metrics: {"child_traces":[["op",{"apply.queue_time_us":236,"mutex_wait_us":350,"prepare.queue_time_us":1637,"prepare.run_cpu_time_us":128,"prepare.run_wall_time_us":551,"replication_time_us":1289995,"thread_start_us":75,"threads_started":1}]]}
I20250111 09:02:10.460695 18935 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=5104, attempt_no=0}) took 1290 ms. Trace:
I20250111 09:02:10.460744 18935 rpcz_store.cc:276] 0111 09:02:09.170568 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:09.170597 (+ 29us) service_pool.cc:225] Handling call
0111 09:02:10.460684 (+1290087us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:09.172238 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 6ab63c6543ad4a0ab83e76267a725edd
0111 09:02:09.172297 (+ 59us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:09.172299 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:02:09.172300 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:09.172337 (+ 37us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:02:09.172360 (+ 23us) tablet.cc:698] Row locks acquired
0111 09:02:09.172361 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:02:09.172384 (+ 23us) write_op.cc:270] Start()
0111 09:02:09.172400 (+ 16us) write_op.cc:276] Timestamp: P: 1736586129172311 usec, L: 0
0111 09:02:09.172401 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:02:09.172486 (+ 85us) log.cc:844] Serialized 307 byte log entry
0111 09:02:10.456685 (+1284199us) op_driver.cc:464] REPLICATION: finished
0111 09:02:10.456995 (+ 310us) write_op.cc:301] APPLY: starting
0111 09:02:10.460352 (+ 3357us) spinlock_profiling.cc:243] Waited 1.05 ms on lock 0x558c478665d8. stack: 00007f3e831c61af 00007f3e831c67a1 00007f3e82d2e89d 0000558c2a807bee 0000558c2a807c41 0000558c2a80bf7a 00007f3e852f40ef 00007f3e852f6f68 00007f3e8526ff15 00007f3e8519e5e4 00007f3e851a0eb6 00007f3e8526e36a 00007f3e8525ae43 00007f3e8525a356 00007f3e8525d2e7 0000558c2a894bbd
0111 09:02:10.460361 (+ 9us) tablet.cc:1364] starting BulkCheckPresence
0111 09:02:10.460493 (+ 132us) tablet.cc:1367] finished BulkCheckPresence
0111 09:02:10.460495 (+ 2us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:02:10.460525 (+ 30us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:02:10.460546 (+ 21us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=81
0111 09:02:10.460550 (+ 4us) write_op.cc:312] APPLY: finished
0111 09:02:10.460586 (+ 36us) log.cc:844] Serialized 27 byte log entry
0111 09:02:10.460626 (+ 40us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:10.460634 (+ 8us) write_op.cc:454] Released schema lock
0111 09:02:10.460637 (+ 3us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":276,"num_ops":1,"prepare.queue_time_us":1433,"prepare.run_cpu_time_us":289,"prepare.run_wall_time_us":445,"replication_time_us":1284272,"spinlock_wait_cycles":2418432,"thread_start_us":63,"threads_started":1}]]}
W20250111 09:02:10.463632 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:10.465220 18936 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.ParticipateInTransaction from 127.14.61.3:36423 (ReqId={client: c602c61032fa436092728430b0406df9, seq_no=3284, attempt_no=0}) took 1300 ms. Trace:
I20250111 09:02:10.465343 18936 rpcz_store.cc:276] 0111 09:02:09.164718 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:09.164751 (+ 33us) service_pool.cc:225] Handling call
0111 09:02:10.465206 (+1300455us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:09.165658 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:09.165683 (+ 25us) participant_op.cc:199] PREPARE: finished
0111 09:02:09.165698 (+ 15us) participant_op.cc:212] START. Timestamp: 1736586129165623
0111 09:02:09.165701 (+ 3us) op_driver.cc:348] REPLICATION: starting
0111 09:02:09.165774 (+ 73us) log.cc:844] Serialized 78 byte log entry
0111 09:02:10.456471 (+1290697us) op_driver.cc:464] REPLICATION: finished
0111 09:02:10.456712 (+ 241us) participant_op.cc:265] APPLY: starting
0111 09:02:10.464940 (+ 8228us) participant_op.cc:270] APPLY: finished
0111 09:02:10.464970 (+ 30us) log.cc:844] Serialized 16 byte log entry
0111 09:02:10.465173 (+ 203us) participant_op.cc:94] Released txn lock
0111 09:02:10.465187 (+ 14us) participant_op.cc:291] FINISH: Op applied
Metrics: {"child_traces":[["op",{"apply.queue_time_us":204,"prepare.queue_time_us":688,"prepare.run_cpu_time_us":166,"prepare.run_wall_time_us":841,"raft.queue_time_us":407,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":102,"replication_time_us":1290748,"spinlock_wait_cycles":779136,"thread_start_us":48,"threads_started":1}]]}
W20250111 09:02:10.471617 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:10.475369 18937 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=5099, attempt_no=0}) took 1328 ms. Trace:
I20250111 09:02:10.475469 18937 rpcz_store.cc:276] 0111 09:02:09.146667 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:09.146700 (+ 33us) service_pool.cc:225] Handling call
0111 09:02:10.475360 (+1328660us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:09.173055 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:09.173073 (+ 18us) participant_op.cc:199] PREPARE: finished
0111 09:02:09.173087 (+ 14us) participant_op.cc:212] START. Timestamp: 1736586129173014
0111 09:02:09.173089 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:02:09.173180 (+ 91us) log.cc:844] Serialized 68 byte log entry
0111 09:02:10.456792 (+1283612us) op_driver.cc:464] REPLICATION: finished
0111 09:02:10.458513 (+ 1721us) participant_op.cc:265] APPLY: starting
0111 09:02:10.471276 (+ 12763us) participant_op.cc:270] APPLY: finished
0111 09:02:10.471299 (+ 23us) log.cc:844] Serialized 16 byte log entry
0111 09:02:10.471334 (+ 35us) participant_op.cc:94] Released txn lock
0111 09:02:10.471341 (+ 7us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:02:10.471993 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 6ab63c6543ad4a0ab83e76267a725edd
0111 09:02:10.472033 (+ 40us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:10.472034 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:02:10.472035 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:10.472061 (+ 26us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:02:10.472079 (+ 18us) tablet.cc:698] Row locks acquired
0111 09:02:10.472080 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:02:10.472095 (+ 15us) write_op.cc:270] Start()
0111 09:02:10.472107 (+ 12us) write_op.cc:276] Timestamp: P: 1736586130472070 usec, L: 0
0111 09:02:10.472108 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:02:10.472165 (+ 57us) log.cc:844] Serialized 307 byte log entry
0111 09:02:10.474883 (+ 2718us) op_driver.cc:464] REPLICATION: finished
0111 09:02:10.474971 (+ 88us) write_op.cc:301] APPLY: starting
0111 09:02:10.474986 (+ 15us) tablet.cc:1364] starting BulkCheckPresence
0111 09:02:10.475139 (+ 153us) tablet.cc:1367] finished BulkCheckPresence
0111 09:02:10.475141 (+ 2us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:02:10.475201 (+ 60us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:02:10.475228 (+ 27us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=82
0111 09:02:10.475233 (+ 5us) write_op.cc:312] APPLY: finished
0111 09:02:10.475272 (+ 39us) log.cc:844] Serialized 27 byte log entry
0111 09:02:10.475300 (+ 28us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:10.475307 (+ 7us) write_op.cc:454] Released schema lock
0111 09:02:10.475311 (+ 4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":15360,"txn-participant-registration.queue_time_us":46,"txn-participant-registration.run_cpu_time_us":311,"txn-participant-registration.run_wall_time_us":26135,"child_traces":[["op",{"apply.queue_time_us":1213,"apply.run_cpu_time_us":1018,"apply.run_wall_time_us":13098,"prepare.queue_time_us":44,"prepare.run_cpu_time_us":163,"prepare.run_wall_time_us":157,"replication_time_us":1283691,"spinlock_wait_cycles":1108224,"thread_start_us":60,"threads_started":1,"child_traces":[["op",{"apply.queue_time_us":52,"num_ops":1,"prepare.queue_time_us":408,"prepare.run_cpu_time_us":209,"prepare.run_wall_time_us":209,"replication_time_us":2768,"spinlock_wait_cycles":28544}]]}]]}
W20250111 09:02:10.479805 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.482954 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.483050 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.483287 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.485733 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.486583 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.488442 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.491748 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:10.493705 17467 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 134 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 133 index: 3963. Preceding OpId from leader: term: 134 index: 3965. (index mismatch)
I20250111 09:02:10.494717 18951 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3964, Last known committed idx: 3963, Time since last communication: 0.000s
I20250111 09:02:10.501590 17336 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 134 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 133 index: 3963. Preceding OpId from leader: term: 134 index: 3965. (index mismatch)
I20250111 09:02:10.504660 18924 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3964, Last known committed idx: 3963, Time since last communication: 0.000s
W20250111 09:02:10.546252 18923 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.548137 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.555493 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.556497 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.557979 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.558454 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.753409 18958 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 68 similar messages]
I20250111 09:02:10.921262 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:02:10.925503 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:10.927359 17340 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 134 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:10.927525 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 134 FOLLOWER]: Advancing to term 135
I20250111 09:02:10.929136 17340 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 135 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:10.929358 17338 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 135 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 134. Current term is 135. Ops: [134.4071-134.4071]
I20250111 09:02:10.930696 18922 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: INVALID_TERM, Last received: 134.4070, Next index: 4071, Last known committed idx: 4067, Time since last communication: 0.000s
I20250111 09:02:10.930863 18922 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 134 LEADER]: Stepping down as leader of term 134
I20250111 09:02:10.930938 18922 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 134 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:10.931066 18922 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4070, Committed index: 4070, Last appended: 134.4071, Last appended by leader: 4071, Current term: 134, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:10.931218 18922 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 134 FOLLOWER]: Advancing to term 135
I20250111 09:02:10.932044 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 135 candidate_status { last_received { term: 134 index: 4070 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:10.932049 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 135 candidate_status { last_received { term: 134 index: 4070 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:10.932153 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 134 FOLLOWER]: Advancing to term 135
I20250111 09:02:10.931751 17340 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 135 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:10.932780 17611 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 135 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 135 because replica has last-logged OpId of term: 134 index: 4071, which is greater than that of the candidate, which has last-logged OpId of term: 134 index: 4070.
I20250111 09:02:10.933166 17476 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 135 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 135 because replica has last-logged OpId of term: 134 index: 4071, which is greater than that of the candidate, which has last-logged OpId of term: 134 index: 4070.
I20250111 09:02:10.933429 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 135 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:10.933614 18930 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 135 FOLLOWER]: Leader election lost for term 135. Reason: could not achieve majority
I20250111 09:02:10.935060 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:10.935899 17340 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 133 FOLLOWER]: Starting forced leader election (received explicit request)
W20250111 09:02:10.935981 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:10.935976 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 133 FOLLOWER]: Advancing to term 134
W20250111 09:02:10.937764 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:10.939875 17340 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 134 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:10.940193 17340 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 134 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:10.940595 17338 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 134 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 133. Current term is 134. Ops: [133.4121-133.4121]
I20250111 09:02:10.940701 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 134 candidate_status { last_received { term: 133 index: 4120 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:10.940800 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 133 FOLLOWER]: Advancing to term 134
W20250111 09:02:10.940825 18894 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 63 similar messages]
W20250111 09:02:10.940888 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:10.941711 17474 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 134 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 134 because replica has last-logged OpId of term: 133 index: 4121, which is greater than that of the candidate, which has last-logged OpId of term: 133 index: 4120.
I20250111 09:02:10.941823 17477 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 134 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 133. Current term is 134. Ops: [133.4122-133.4123]
I20250111 09:02:10.942369 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 134 candidate_status { last_received { term: 133 index: 4120 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:10.942822 18963 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: INVALID_TERM, Last received: 133.4120, Next index: 4121, Last known committed idx: 4120, Time since last communication: 0.000s
I20250111 09:02:10.942883 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 134 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:10.943053 18930 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 134 FOLLOWER]: Leader election lost for term 134. Reason: could not achieve majority
I20250111 09:02:10.943094 18963 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: INVALID_TERM, Last received: 133.4121, Next index: 4122, Last known committed idx: 4121, Time since last communication: 0.000s
I20250111 09:02:10.943485 18951 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 133 LEADER]: Stepping down as leader of term 133
I20250111 09:02:10.943535 18951 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 133 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:10.943655 18951 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4121, Committed index: 4121, Last appended: 133.4124, Last appended by leader: 4124, Current term: 133, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:10.944116 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:10.944471 18951 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 133 FOLLOWER]: Advancing to term 134
W20250111 09:02:10.944525 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:10.944623 17340 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 134 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 133. Current term is 134. Ops: [133.4121-133.4124]
W20250111 09:02:10.946785 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.948113 18893 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 61 similar messages]
W20250111 09:02:10.948165 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.949888 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.949972 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.951125 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.954015 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.954902 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.956828 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.957000 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.962864 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.962990 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.964815 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.966224 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.968716 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.971351 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.972934 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.975668 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.978930 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.980070 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.983829 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.986202 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.987830 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.989428 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:10.995406 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.997694 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.997768 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:10.999554 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.005334 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.012197 18923 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.012228 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.012265 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.019057 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.024451 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.026417 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.027741 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.032665 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.036585 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.038836 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.040881 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.045696 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.052613 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.056245 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.058478 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.062288 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.071954 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.072721 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.076429 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.080247 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:11.081517 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:02:11.081929 17474 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:11.082032 17474 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 135 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:11.082104 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 135 FOLLOWER]: Advancing to term 136
I20250111 09:02:11.083024 17474 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 136 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:11.083449 17474 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 136 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:11.083755 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 136 candidate_status { last_received { term: 134 index: 4071 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:11.083784 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 136 candidate_status { last_received { term: 134 index: 4071 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:11.083895 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 135 FOLLOWER]: Advancing to term 136
I20250111 09:02:11.083897 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 135 FOLLOWER]: Advancing to term 136
I20250111 09:02:11.084100 17474 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:11.084215 17474 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 134 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:11.084266 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 134 FOLLOWER]: Advancing to term 135
I20250111 09:02:11.084779 17340 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 136 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 136.
I20250111 09:02:11.084913 17611 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 136 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 136.
I20250111 09:02:11.085146 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 136 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:11.085036 17474 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 135 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:11.085481 18966 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 136 FOLLOWER]: Leader election won for term 136
I20250111 09:02:11.085549 17474 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 135 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:11.085664 18966 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 136 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:11.085829 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 135 candidate_status { last_received { term: 133 index: 4121 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:11.085815 18966 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4070, Committed index: 4070, Last appended: 134.4071, Last appended by leader: 4070, Current term: 136, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:11.085923 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 134 FOLLOWER]: Advancing to term 135
I20250111 09:02:11.086416 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 135 candidate_status { last_received { term: 133 index: 4121 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:11.086498 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 134 FOLLOWER]: Advancing to term 135
I20250111 09:02:11.086635 17340 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 135 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 135.
I20250111 09:02:11.086870 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 135 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:11.087157 18966 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 135 FOLLOWER]: Leader election won for term 135
I20250111 09:02:11.087229 18966 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 135 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:11.087138 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 134 to 136, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 136 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:11.087378 17611 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 135 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 135 because replica has last-logged OpId of term: 133 index: 4124, which is greater than that of the candidate, which has last-logged OpId of term: 133 index: 4121.
I20250111 09:02:11.087340 18966 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4121, Committed index: 4121, Last appended: 133.4121, Last appended by leader: 4121, Current term: 135, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:11.089112 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.089949 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:11.090674 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 133 to 135, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 135 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:02:11.094581 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:11.101310 17340 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 136 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 134 index: 4070. Preceding OpId from leader: term: 136 index: 4073. (index mismatch)
I20250111 09:02:11.101545 17611 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 136 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 134 index: 4071. Preceding OpId from leader: term: 136 index: 4073. (index mismatch)
I20250111 09:02:11.101625 18966 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4072, Last known committed idx: 4067, Time since last communication: 0.000s
I20250111 09:02:11.101886 18967 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4072, Last known committed idx: 4070, Time since last communication: 0.000s
W20250111 09:02:11.105804 18937 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:11.109603 17340 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 135 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 133 index: 4120. Preceding OpId from leader: term: 135 index: 4123. (index mismatch)
I20250111 09:02:11.110036 18967 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4122, Last known committed idx: 4120, Time since last communication: 0.000s
I20250111 09:02:11.110091 17611 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 135 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 133 index: 4124. Preceding OpId from leader: term: 135 index: 4123. (term mismatch)
I20250111 09:02:11.110155 17611 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 4122
I20250111 09:02:11.110177 17611 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 133.4123
W20250111 09:02:11.110248 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:02:11.110379 17611 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 133.4124
W20250111 09:02:11.110460 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:02:11.111263 18967 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4122, Last known committed idx: 4121, Time since last communication: 0.000s
I20250111 09:02:11.111356 18967 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 133.4122 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4121
I20250111 09:02:11.112036 17612 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 4121
I20250111 09:02:11.112085 17612 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 133.4122
W20250111 09:02:11.112161 17612 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
W20250111 09:02:11.136250 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.136651 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.140640 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.142614 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:11.372936 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:02:11.379912 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:11.380640 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 136 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:11.381374 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 136 FOLLOWER]: Advancing to term 137
I20250111 09:02:11.382987 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 137 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:11.391392 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 137 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:11.391841 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 137 candidate_status { last_received { term: 136 index: 4133 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:11.391886 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:11.391978 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 135 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:11.392028 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 135 FOLLOWER]: Advancing to term 136
I20250111 09:02:11.394279 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 136 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:11.394438 17612 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 136 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 135. Current term is 136. Ops: [135.4197-135.4197]
I20250111 09:02:11.394430 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 137 candidate_status { last_received { term: 136 index: 4133 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:11.394570 17477 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 136 LEADER]: Stepping down as leader of term 136
I20250111 09:02:11.394624 17477 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 136 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:11.394766 17477 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4133, Committed index: 4133, Last appended: 136.4137, Last appended by leader: 4137, Current term: 136, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:11.394881 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 136 FOLLOWER]: Advancing to term 137
W20250111 09:02:11.395008 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:11.395042 18971 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: INVALID_TERM, Last received: 135.4196, Next index: 4197, Last known committed idx: 4195, Time since last communication: 0.000s
I20250111 09:02:11.395231 18972 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 135 LEADER]: Stepping down as leader of term 135
I20250111 09:02:11.395272 18972 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 135 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:11.395385 18972 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4196, Committed index: 4196, Last appended: 135.4197, Last appended by leader: 4197, Current term: 135, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:11.395618 18972 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 135 FOLLOWER]: Advancing to term 136
I20250111 09:02:11.395802 17477 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 137 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 137 because replica has last-logged OpId of term: 136 index: 4137, which is greater than that of the candidate, which has last-logged OpId of term: 136 index: 4133.
I20250111 09:02:11.396495 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 137 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:11.396724 18964 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 137 FOLLOWER]: Leader election lost for term 137. Reason: could not achieve majority
W20250111 09:02:11.397493 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.397581 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:11.397642 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 136 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:11.398381 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 136 candidate_status { last_received { term: 135 index: 4196 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:11.398756 17477 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 136 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 136 because replica has last-logged OpId of term: 135 index: 4197, which is greater than that of the candidate, which has last-logged OpId of term: 135 index: 4196.
I20250111 09:02:11.399063 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 136 candidate_status { last_received { term: 135 index: 4196 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:11.399166 17335 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 135 FOLLOWER]: Advancing to term 136
W20250111 09:02:11.399688 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:11.400100 17335 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 136 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 136 because replica has last-logged OpId of term: 135 index: 4197, which is greater than that of the candidate, which has last-logged OpId of term: 135 index: 4196.
I20250111 09:02:11.400352 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 136 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:11.400497 18964 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 136 FOLLOWER]: Leader election lost for term 136. Reason: could not achieve majority
W20250111 09:02:11.400825 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:11.401875 17611 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 137 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 136. Current term is 137. Ops: [136.4134-136.4134]
W20250111 09:02:11.402251 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.403195 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.403285 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.403805 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.405817 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.408648 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.408738 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.409533 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.410063 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.410324 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.412845 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.413648 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.414283 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.414752 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.415079 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.416395 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.417456 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.417564 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.420377 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.421022 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.421902 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.422739 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.423592 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.424634 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.425258 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.425535 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.426597 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.428954 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.429366 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.430763 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.431490 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.433373 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.433547 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.433631 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.434944 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.437579 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.438516 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.439229 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.440369 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.444294 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.444584 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.446192 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.446226 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.447096 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.447566 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.450464 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.451020 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.454538 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.454632 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.454891 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.455348 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.455365 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.457793 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.460520 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.462152 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.463696 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.464917 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.466063 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.466406 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.466485 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.471103 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.471974 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.475090 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.477339 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.477345 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.477409 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.479121 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.481099 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.483470 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.485226 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.487080 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.489174 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.490195 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.492210 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.493615 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.493623 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.497810 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.498387 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.499725 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.503257 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.504921 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.506513 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.509032 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.511255 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.513896 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.514453 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.515769 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.516943 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.519922 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.520707 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.523026 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.527331 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.528756 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.529206 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.532080 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.533442 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.534003 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.535445 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.538776 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.543982 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.546011 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.547921 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.548050 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.548156 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.549968 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.550155 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.553974 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.561316 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.562714 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.565416 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.565902 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.566766 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.567215 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.568506 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.573216 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.580813 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.582584 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.583937 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.584673 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.584853 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.585424 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.587855 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.592965 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.600045 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.600075 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.603180 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.603250 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.603590 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.605353 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.608114 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.614259 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:11.618979 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:02:11.619412 17336 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
W20250111 09:02:11.619489 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:11.619505 17336 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 136 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:11.619557 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 136 FOLLOWER]: Advancing to term 137
I20250111 09:02:11.620406 17336 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 137 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:11.620723 17336 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 137 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:11.620977 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 137 candidate_status { last_received { term: 136 index: 4132 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:11.621134 17611 raft_consensus.cc:2388] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 137 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 in current term 137: Already voted for candidate b5fb421fa3354606aecbeafe55a0565c in this term.
I20250111 09:02:11.621239 17336 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:11.621371 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 137 candidate_status { last_received { term: 136 index: 4132 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:11.621498 17477 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 137 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 137 because replica has last-logged OpId of term: 136 index: 4137, which is greater than that of the candidate, which has last-logged OpId of term: 136 index: 4132.
W20250111 09:02:11.621652 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:11.621723 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 137 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:11.622134 17336 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 136 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:11.622190 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 136 FOLLOWER]: Advancing to term 137
I20250111 09:02:11.622304 18976 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 137 FOLLOWER]: Leader election lost for term 137. Reason: could not achieve majority
W20250111 09:02:11.622874 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:11.622857 17336 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 137 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:11.623131 17336 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 137 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:11.623543 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 137 candidate_status { last_received { term: 135 index: 4197 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:11.623636 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 136 FOLLOWER]: Advancing to term 137
W20250111 09:02:11.623744 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:11.623967 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 137 candidate_status { last_received { term: 135 index: 4197 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:11.624081 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 136 FOLLOWER]: Advancing to term 137
I20250111 09:02:11.624586 17477 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 137 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 137.
I20250111 09:02:11.624816 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 137 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:11.624847 17611 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 137 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 137.
I20250111 09:02:11.624980 18976 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 137 FOLLOWER]: Leader election won for term 137
I20250111 09:02:11.625371 18976 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 137 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:11.625571 18976 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4196, Committed index: 4196, Last appended: 135.4197, Last appended by leader: 4197, Current term: 137, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:11.625700 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Service unavailable: leader is not yet ready
W20250111 09:02:11.626340 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:11.626770 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 135 to 137, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 137 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:02:11.627995 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:11.636016 17477 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 137 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 135 index: 4197. Preceding OpId from leader: term: 137 index: 4199. (index mismatch)
I20250111 09:02:11.636016 17611 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 137 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 135 index: 4196. Preceding OpId from leader: term: 137 index: 4199. (index mismatch)
I20250111 09:02:11.636324 18977 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4198, Last known committed idx: 4195, Time since last communication: 0.000s
I20250111 09:02:11.636440 18976 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4198, Last known committed idx: 4196, Time since last communication: 0.000s
W20250111 09:02:11.641958 18940 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.644215 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.644345 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.646199 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.648445 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.650069 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.651036 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.658092 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.664299 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.664880 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.669569 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.669924 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.674304 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.675014 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.679225 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.681186 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.686555 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.686975 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.686987 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.693753 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.694101 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.694896 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.696266 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.697432 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.700023 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.700701 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.701509 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.706135 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.708222 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.710695 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.714162 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.714942 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.718647 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.719785 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.721690 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.724072 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.725299 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.726796 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.730177 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.731355 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.735906 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.737771 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.740052 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.743917 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.746598 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.747187 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.750411 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.750516 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.751649 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.756152 18956 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 70 similar messages]
W20250111 09:02:11.756199 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.757323 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.762872 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.765062 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.767611 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.772838 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.774034 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.777513 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.780175 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.781124 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.781440 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.791221 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.792527 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.795760 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.798724 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.803499 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.806222 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.806707 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.809127 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.810714 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.816915 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.820209 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.824029 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.828358 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.832026 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.835130 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.837219 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.837985 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.842098 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.846812 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.847139 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.852615 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.862118 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.862685 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.864715 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.869143 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.869716 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.873416 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.877054 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.883579 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.884443 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.892959 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.895179 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.898631 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.902864 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.906567 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.908816 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.909024 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.913828 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.925551 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.929265 18921 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.929482 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.933146 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.933971 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.936730 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.939874 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.952147 18911 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 76 similar messages]
W20250111 09:02:11.952193 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.956913 18918 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 67 similar messages]
W20250111 09:02:11.956977 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.958207 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.958232 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.960268 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.970366 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.974887 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.976763 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.982302 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:11.984398 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.993991 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:11.994542 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.002714 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.003129 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.007640 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.007825 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.010641 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.027866 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.029820 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.030563 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.031898 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.036639 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.037441 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.040629 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.056262 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.059770 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.063876 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.065145 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.066642 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.076328 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.077323 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:12.077415 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:02:12.077936 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:12.078047 17477 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 137 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:12.078101 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 137 FOLLOWER]: Advancing to term 138
I20250111 09:02:12.078924 17477 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 138 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:12.079349 17477 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 138 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:12.079563 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 138 candidate_status { last_received { term: 136 index: 4137 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:12.079672 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 137 FOLLOWER]: Advancing to term 138
I20250111 09:02:12.079815 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 138 candidate_status { last_received { term: 136 index: 4137 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:12.079919 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 137 FOLLOWER]: Advancing to term 138
I20250111 09:02:12.080577 17336 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 138 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 138.
I20250111 09:02:12.080834 17611 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 138 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 138.
I20250111 09:02:12.080828 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 138 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:12.080994 18972 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 138 FOLLOWER]: Leader election won for term 138
I20250111 09:02:12.081167 18972 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 138 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:12.081175 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:12.081295 17477 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 137 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:12.081350 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 137 FOLLOWER]: Advancing to term 138
I20250111 09:02:12.081326 18972 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4133, Committed index: 4133, Last appended: 136.4137, Last appended by leader: 4137, Current term: 138, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:12.081938 17477 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 138 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:12.082217 17477 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 138 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:12.082490 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 138 candidate_status { last_received { term: 137 index: 4208 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:12.082511 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 138 candidate_status { last_received { term: 137 index: 4208 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:12.082587 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 137 FOLLOWER]: Advancing to term 138
I20250111 09:02:12.082587 17336 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 137 LEADER]: Stepping down as leader of term 137
I20250111 09:02:12.082626 17336 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 137 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:12.082754 17336 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4208, Committed index: 4208, Last appended: 137.4208, Last appended by leader: 4208, Current term: 137, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:12.082888 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 137 FOLLOWER]: Advancing to term 138
I20250111 09:02:12.082808 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 136 to 138. New cstate: current_term: 138 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:12.083361 17611 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 138 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 138.
I20250111 09:02:12.083700 17336 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 138 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 138.
I20250111 09:02:12.083920 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 138 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:12.084065 18972 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 138 FOLLOWER]: Leader election won for term 138
I20250111 09:02:12.084132 18972 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 138 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:12.084223 18972 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4208, Committed index: 4208, Last appended: 137.4208, Last appended by leader: 4208, Current term: 138, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:12.084759 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:12.085975 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 137 to 138, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 138 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:12.089315 17611 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 138 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 136 index: 4133. Preceding OpId from leader: term: 138 index: 4139. (index mismatch)
I20250111 09:02:12.089356 17336 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 138 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 136 index: 4132. Preceding OpId from leader: term: 138 index: 4139. (index mismatch)
I20250111 09:02:12.089607 18972 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4138, Last known committed idx: 4133, Time since last communication: 0.000s
I20250111 09:02:12.089716 18983 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4138, Last known committed idx: 4127, Time since last communication: 0.000s
W20250111 09:02:12.103067 18918 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.103879 18992 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.105743 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.106285 18991 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.111717 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:12.111761 17611 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 138 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 137 index: 4208. Preceding OpId from leader: term: 138 index: 4210. (index mismatch)
I20250111 09:02:12.112028 17338 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 138 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 137 index: 4208. Preceding OpId from leader: term: 138 index: 4210. (index mismatch)
I20250111 09:02:12.112247 18994 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4209, Last known committed idx: 4208, Time since last communication: 0.000s
I20250111 09:02:12.112468 18983 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4209, Last known committed idx: 4208, Time since last communication: 0.000s
W20250111 09:02:12.114774 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.131575 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.131870 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.139194 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.142985 18956 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.150396 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.154510 18991 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.155200 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.155870 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:12.157765 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:02:12.159031 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:12.159137 17611 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 138 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:12.159185 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 138 FOLLOWER]: Advancing to term 139
I20250111 09:02:12.160085 17611 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 139 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:12.160390 17611 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 139 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:12.160936 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 139 candidate_status { last_received { term: 138 index: 4153 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:12.161064 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 138 FOLLOWER]: Advancing to term 139
I20250111 09:02:12.161885 17340 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 139 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 139.
I20250111 09:02:12.162119 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 139 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:12.162326 18963 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 139 FOLLOWER]: Leader election won for term 139
I20250111 09:02:12.162397 18963 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 139 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:12.162483 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 139 candidate_status { last_received { term: 138 index: 4153 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:12.162504 18963 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4153, Committed index: 4153, Last appended: 138.4153, Last appended by leader: 4153, Current term: 139, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:12.162571 17474 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 138 LEADER]: Stepping down as leader of term 138
I20250111 09:02:12.162604 17474 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 138 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:12.162698 17474 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4153, Committed index: 4153, Last appended: 138.4153, Last appended by leader: 4153, Current term: 138, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:12.162806 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 138 FOLLOWER]: Advancing to term 139
I20250111 09:02:12.163630 17474 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 139 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 139.
W20250111 09:02:12.164415 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:12.164718 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:12.164809 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 138 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:12.164851 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 138 FOLLOWER]: Advancing to term 139
W20250111 09:02:12.164923 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:12.166210 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 139 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:12.166409 17611 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 139 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 138. Current term is 139. Ops: [138.4225-138.4225]
I20250111 09:02:12.166993 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 138 to 139, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 139 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:12.167093 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 139 candidate_status { last_received { term: 138 index: 4224 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:12.167100 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 139 candidate_status { last_received { term: 138 index: 4224 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:12.167208 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 138 FOLLOWER]: Advancing to term 139
I20250111 09:02:12.166755 18993 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: INVALID_TERM, Last received: 138.4224, Next index: 4225, Last known committed idx: 4223, Time since last communication: 0.000s
I20250111 09:02:12.167537 17477 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 138 LEADER]: Stepping down as leader of term 138
I20250111 09:02:12.167610 17477 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 138 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:12.167750 17477 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4224, Committed index: 4224, Last appended: 138.4225, Last appended by leader: 4225, Current term: 138, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:12.167894 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 138 FOLLOWER]: Advancing to term 139
I20250111 09:02:12.168077 17340 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 139 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 139.
I20250111 09:02:12.168244 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 139 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:12.168488 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 139 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:12.168795 18964 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 139 FOLLOWER]: Leader election won for term 139
I20250111 09:02:12.168824 17477 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 139 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 139 because replica has last-logged OpId of term: 138 index: 4225, which is greater than that of the candidate, which has last-logged OpId of term: 138 index: 4224.
I20250111 09:02:12.168884 18964 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 139 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
W20250111 09:02:12.168913 18983 raft_consensus.cc:931] Couldn't advance consensus term.: Illegal state: Can't advance term to: 139 current term: 139 is higher.
I20250111 09:02:12.168988 18964 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4223, Committed index: 4223, Last appended: 138.4224, Last appended by leader: 4224, Current term: 139, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:12.170251 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 138 to 139, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 139 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
W20250111 09:02:12.170647 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:12.171453 17340 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 139 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 138. Current term is 139. Ops: [138.4224-138.4225]
W20250111 09:02:12.172350 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:12.173513 17474 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 139 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 138 index: 4153. Preceding OpId from leader: term: 139 index: 4155. (index mismatch)
W20250111 09:02:12.173676 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:12.174022 18964 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4154, Last known committed idx: 4153, Time since last communication: 0.000s
W20250111 09:02:12.174985 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.178488 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:12.178784 17338 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 139 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 138 index: 4153. Preceding OpId from leader: term: 139 index: 4155. (index mismatch)
I20250111 09:02:12.179128 18951 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4154, Last known committed idx: 4153, Time since last communication: 0.000s
I20250111 09:02:12.183403 17474 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 139 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 138 index: 4225. Preceding OpId from leader: term: 139 index: 4226. (index mismatch)
W20250111 09:02:12.183512 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:12.183665 17338 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 139 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 138 index: 4223. Preceding OpId from leader: term: 139 index: 4226. (index mismatch)
I20250111 09:02:12.183991 18963 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4225, Last known committed idx: 4224, Time since last communication: 0.000s
I20250111 09:02:12.184067 18963 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer 262623ac6df745c785ce267ea39bd265 log is divergent from this leader: its last log entry 138.4225 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4224
I20250111 09:02:12.185519 18963 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4225, Last known committed idx: 4220, Time since last communication: 0.000s
I20250111 09:02:12.189860 17477 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 4224
I20250111 09:02:12.189940 17477 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 138.4225
W20250111 09:02:12.190073 17477 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
I20250111 09:02:12.294631 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:02:12.295795 17335 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:12.296491 17335 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 139 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:12.296569 17335 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 139 FOLLOWER]: Advancing to term 140
I20250111 09:02:12.297627 17335 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 140 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:12.298019 17335 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 140 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:12.298308 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 140 candidate_status { last_received { term: 139 index: 4186 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:12.299006 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 140 candidate_status { last_received { term: 139 index: 4186 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:12.299119 17614 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 139 LEADER]: Stepping down as leader of term 139
I20250111 09:02:12.299171 17614 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 139 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:12.299408 17614 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4184, Committed index: 4184, Last appended: 139.4187, Last appended by leader: 4187, Current term: 139, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:12.299505 17336 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:12.299578 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 139 FOLLOWER]: Advancing to term 140
I20250111 09:02:12.299590 17336 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 139 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:12.299649 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 139 FOLLOWER]: Advancing to term 140
I20250111 09:02:12.300390 17336 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 140 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:12.300457 17614 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 140 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 140 because replica has last-logged OpId of term: 139 index: 4187, which is greater than that of the candidate, which has last-logged OpId of term: 139 index: 4186.
I20250111 09:02:12.300696 17336 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 140 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:12.300916 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 140 candidate_status { last_received { term: 139 index: 4250 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
W20250111 09:02:12.301101 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:12.301360 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 140 candidate_status { last_received { term: 139 index: 4250 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:12.301426 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 140 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:12.301450 17614 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 139 LEADER]: Stepping down as leader of term 139
I20250111 09:02:12.301486 17614 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 139 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:12.301584 17614 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4256, Committed index: 4256, Last appended: 139.4257, Last appended by leader: 4257, Current term: 139, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:12.301617 18981 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 140 FOLLOWER]: Leader election lost for term 140. Reason: could not achieve majority
I20250111 09:02:12.301667 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 139 FOLLOWER]: Advancing to term 140
W20250111 09:02:12.301851 18998 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.008s ago.
W20250111 09:02:12.302232 18988 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:12.302457 17614 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 140 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 140 because replica has last-logged OpId of term: 139 index: 4257, which is greater than that of the candidate, which has last-logged OpId of term: 139 index: 4250.
W20250111 09:02:12.302776 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:12.302938 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 140 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:12.303093 17338 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 140 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 139. Current term is 140. Ops: [139.4251-139.4252]
I20250111 09:02:12.303411 18981 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 140 FOLLOWER]: Leader election lost for term 140. Reason: could not achieve majority
W20250111 09:02:12.305826 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.307137 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.308387 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.309998 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.310350 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.311512 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.313642 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.316510 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.316804 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.317551 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.318519 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.323071 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.325733 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.327026 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.327080 18991 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.331276 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.335284 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.338235 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.338285 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.339402 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.347232 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.347776 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.347822 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.349187 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.356768 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.359066 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.360078 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.361182 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.367744 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.369901 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.371058 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.374161 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.380873 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.384552 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.384754 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.387611 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.394335 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.397423 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.400633 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.403940 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.408779 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.414703 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.417899 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.419270 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.426950 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.428421 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.435945 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.437086 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.442603 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.443825 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.452056 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.454345 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.463027 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.463027 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.470106 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.474666 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.481567 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.481889 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.488608 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.496232 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.502172 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.503433 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.506438 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.515573 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.522178 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.524322 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.525588 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.538861 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.543599 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.546298 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.547765 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.562321 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.567003 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.569137 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.571161 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.583977 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.590623 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.592774 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.594940 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.609303 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.616189 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.617789 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.618093 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.632835 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.640625 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.643661 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.645838 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.659957 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.665740 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.669772 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.673872 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.689493 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.695096 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.695894 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.701498 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.717275 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.722081 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.725153 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.731432 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.745515 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.754164 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.757329 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.760552 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.776688 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.784435 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.790163 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.790457 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.809253 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.818246 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.822175 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.823685 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.841532 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.853443 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.854398 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.856397 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.874356 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.886266 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.888962 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.890278 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.909066 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.919750 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.921743 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.923882 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.943136 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.953900 18911 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 61 similar messages]
W20250111 09:02:12.953949 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.957872 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.959146 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.977232 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:12.992218 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.993245 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:12.995827 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.014526 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.030364 18894 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 62 similar messages]
W20250111 09:02:13.030416 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.032300 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.033465 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.053758 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.071599 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.071599 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.074523 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.092936 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.111986 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.112072 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.114609 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.134312 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.153450 18894 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.154738 19009 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.154780 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.176550 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.196867 18893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.197100 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.197726 19010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.220853 19010 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.241034 17574 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.241545 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.243374 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:13.262344 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:02:13.262799 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:13.262918 17476 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 139 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:13.262980 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 139 FOLLOWER]: Advancing to term 140
I20250111 09:02:13.263762 17476 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 140 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:13.264158 17476 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 140 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:13.264401 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 140 candidate_status { last_received { term: 139 index: 4187 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:13.264540 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 140 candidate_status { last_received { term: 139 index: 4187 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:13.264583 17338 raft_consensus.cc:2388] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 140 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 in current term 140: Already voted for candidate fb270e42d7c9400f8c05bb9565db1972 in this term.
I20250111 09:02:13.265553 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 140 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 140.
I20250111 09:02:13.265681 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:13.265789 17476 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 139 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:13.265846 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 139 FOLLOWER]: Advancing to term 140
I20250111 09:02:13.265853 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 140 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:13.266135 19011 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 140 FOLLOWER]: Leader election won for term 140
I20250111 09:02:13.266284 19011 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 140 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:13.266438 19011 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4184, Committed index: 4184, Last appended: 139.4187, Last appended by leader: 4187, Current term: 140, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:13.266666 17476 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 140 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:13.266966 17476 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 140 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:13.267257 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 140 candidate_status { last_received { term: 139 index: 4257 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:13.267390 17338 raft_consensus.cc:2388] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 140 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 in current term 140: Already voted for candidate fb270e42d7c9400f8c05bb9565db1972 in this term.
I20250111 09:02:13.267493 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 140 candidate_status { last_received { term: 139 index: 4257 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:13.268098 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 139 to 140, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 140 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:13.268316 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 140 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 140.
W20250111 09:02:13.268682 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:13.268765 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 140 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:13.268926 19011 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 140 FOLLOWER]: Leader election won for term 140
I20250111 09:02:13.269045 19011 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 140 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:13.269169 19011 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4256, Committed index: 4256, Last appended: 139.4257, Last appended by leader: 4257, Current term: 140, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:13.271603 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 139 to 140, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 140 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:02:13.283644 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.287673 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.289898 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.315210 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:13.331173 17338 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 140 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 139 index: 4250. Preceding OpId from leader: term: 140 index: 4259. (index mismatch)
I20250111 09:02:13.331204 17614 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 140 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 139 index: 4257. Preceding OpId from leader: term: 140 index: 4259. (index mismatch)
I20250111 09:02:13.331537 19012 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4258, Last known committed idx: 4248, Time since last communication: 0.000s
I20250111 09:02:13.331739 19011 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4258, Last known committed idx: 4256, Time since last communication: 0.000s
I20250111 09:02:13.336160 17614 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 140 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 139 index: 4187. Preceding OpId from leader: term: 140 index: 4189. (index mismatch)
I20250111 09:02:13.336542 19012 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4188, Last known committed idx: 4184, Time since last communication: 0.000s
W20250111 09:02:13.337664 19029 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:13.337826 19029 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=5420, attempt_no=0}) took 1052 ms. Trace:
I20250111 09:02:13.337862 19029 rpcz_store.cc:276] 0111 09:02:12.285744 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:12.285847 (+ 103us) service_pool.cc:225] Handling call
0111 09:02:13.337820 (+1051973us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:12.293873 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:12.293889 (+ 16us) participant_op.cc:199] PREPARE: finished
0111 09:02:12.293907 (+ 18us) participant_op.cc:212] START. Timestamp: 1736586132293856
0111 09:02:12.293909 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:02:12.294003 (+ 94us) log.cc:844] Serialized 68 byte log entry
0111 09:02:13.334300 (+1040297us) op_driver.cc:464] REPLICATION: finished
0111 09:02:13.336408 (+ 2108us) participant_op.cc:265] APPLY: starting
0111 09:02:13.337184 (+ 776us) participant_op.cc:270] APPLY: finished
0111 09:02:13.337217 (+ 33us) log.cc:844] Serialized 16 byte log entry
0111 09:02:13.337245 (+ 28us) participant_op.cc:94] Released txn lock
0111 09:02:13.337250 (+ 5us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:02:13.337643 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER)
0111 09:02:13.337648 (+ 5us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:13.337650 (+ 2us) write_op.cc:454] Released schema lock
0111 09:02:13.337652 (+ 2us) write_op.cc:335] FINISH: Op aborted
Metrics: {"spinlock_wait_cycles":49024,"txn-participant-registration.queue_time_us":1417,"txn-participant-registration.run_cpu_time_us":287,"txn-participant-registration.run_wall_time_us":6385,"child_traces":[["op",{"apply.queue_time_us":2071,"prepare.queue_time_us":26,"prepare.run_cpu_time_us":167,"prepare.run_wall_time_us":341,"replication_time_us":1040373,"thread_start_us":63,"threads_started":1,"child_traces":[["op",{}]]}]]}
I20250111 09:02:13.338780 17340 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 140 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 139 index: 4186. Preceding OpId from leader: term: 140 index: 4189. (index mismatch)
I20250111 09:02:13.339372 19031 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4188, Last known committed idx: 4184, Time since last communication: 0.000s
I20250111 09:02:13.340988 19017 mvcc.cc:204] Tried to move back new op lower bound from 7113056802139885568 to 7113056801860714496. Current Snapshot: MvccSnapshot[applied={T|T < 7113056797869744128}]
I20250111 09:02:13.342239 19029 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=5422, attempt_no=0}) took 1050 ms. Trace:
I20250111 09:02:13.342285 19029 rpcz_store.cc:276] 0111 09:02:12.291743 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:12.291776 (+ 33us) service_pool.cc:225] Handling call
0111 09:02:13.342231 (+1050455us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:12.292329 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 2a4e17d0e2894eaa95b650f1af0ce650
0111 09:02:12.292383 (+ 54us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:12.292385 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:02:12.292386 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:12.292421 (+ 35us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:02:12.292443 (+ 22us) tablet.cc:698] Row locks acquired
0111 09:02:12.292444 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:02:12.292465 (+ 21us) write_op.cc:270] Start()
0111 09:02:12.292491 (+ 26us) write_op.cc:276] Timestamp: P: 1736586132292418 usec, L: 0
0111 09:02:12.292492 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:02:12.292581 (+ 89us) log.cc:844] Serialized 307 byte log entry
0111 09:02:13.341798 (+1049217us) op_driver.cc:464] REPLICATION: finished
0111 09:02:13.341840 (+ 42us) write_op.cc:301] APPLY: starting
0111 09:02:13.341853 (+ 13us) tablet.cc:1364] starting BulkCheckPresence
0111 09:02:13.342050 (+ 197us) tablet.cc:1367] finished BulkCheckPresence
0111 09:02:13.342051 (+ 1us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:02:13.342079 (+ 28us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:02:13.342110 (+ 31us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=88
0111 09:02:13.342114 (+ 4us) write_op.cc:312] APPLY: finished
0111 09:02:13.342148 (+ 34us) log.cc:844] Serialized 27 byte log entry
0111 09:02:13.342172 (+ 24us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:13.342178 (+ 6us) write_op.cc:454] Released schema lock
0111 09:02:13.342181 (+ 3us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":17,"num_ops":1,"prepare.queue_time_us":321,"prepare.run_cpu_time_us":291,"prepare.run_wall_time_us":290,"replication_time_us":1049294}]]}
I20250111 09:02:13.342356 19036 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=5423, attempt_no=0}) took 1048 ms. Trace:
I20250111 09:02:13.343784 19036 rpcz_store.cc:276] 0111 09:02:12.293400 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:12.293648 (+ 248us) service_pool.cc:225] Handling call
0111 09:02:13.342350 (+1048702us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:12.293938 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 2a4e17d0e2894eaa95b650f1af0ce650
0111 09:02:12.294011 (+ 73us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:12.294015 (+ 4us) write_op.cc:435] Acquired schema lock
0111 09:02:12.294017 (+ 2us) tablet.cc:659] Decoding operations
0111 09:02:12.294061 (+ 44us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:02:12.294102 (+ 41us) tablet.cc:698] Row locks acquired
0111 09:02:12.294103 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:02:12.294124 (+ 21us) write_op.cc:270] Start()
0111 09:02:12.294140 (+ 16us) write_op.cc:276] Timestamp: P: 1736586132294077 usec, L: 0
0111 09:02:12.294143 (+ 3us) op_driver.cc:348] REPLICATION: starting
0111 09:02:12.294236 (+ 93us) log.cc:844] Serialized 307 byte log entry
0111 09:02:13.341835 (+1047599us) op_driver.cc:464] REPLICATION: finished
0111 09:02:13.342094 (+ 259us) write_op.cc:301] APPLY: starting
0111 09:02:13.342104 (+ 10us) tablet.cc:1364] starting BulkCheckPresence
0111 09:02:13.342195 (+ 91us) tablet.cc:1367] finished BulkCheckPresence
0111 09:02:13.342196 (+ 1us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:02:13.342219 (+ 23us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:02:13.342243 (+ 24us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=88
0111 09:02:13.342247 (+ 4us) write_op.cc:312] APPLY: finished
0111 09:02:13.342277 (+ 30us) log.cc:844] Serialized 27 byte log entry
0111 09:02:13.342302 (+ 25us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:13.342307 (+ 5us) write_op.cc:454] Released schema lock
0111 09:02:13.342310 (+ 3us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":223,"num_ops":1,"prepare.queue_time_us":48,"prepare.run_cpu_time_us":364,"prepare.run_wall_time_us":697,"replication_time_us":1047682,"thread_start_us":74,"threads_started":1}]]}
W20250111 09:02:13.344041 19030 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:13.345355 19030 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=5418, attempt_no=0}) took 1062 ms. Trace:
I20250111 09:02:13.345729 19030 rpcz_store.cc:276] 0111 09:02:12.283135 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:12.283167 (+ 32us) service_pool.cc:225] Handling call
0111 09:02:13.345345 (+1062178us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:12.289386 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:12.289399 (+ 13us) participant_op.cc:199] PREPARE: finished
0111 09:02:12.289410 (+ 11us) participant_op.cc:212] START. Timestamp: 1736586132289363
0111 09:02:12.289412 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:02:12.289491 (+ 79us) log.cc:844] Serialized 68 byte log entry
0111 09:02:13.341747 (+1052256us) op_driver.cc:464] REPLICATION: finished
0111 09:02:13.342961 (+ 1214us) participant_op.cc:265] APPLY: starting
0111 09:02:13.343805 (+ 844us) participant_op.cc:270] APPLY: finished
0111 09:02:13.343822 (+ 17us) log.cc:844] Serialized 16 byte log entry
0111 09:02:13.343918 (+ 96us) participant_op.cc:94] Released txn lock
0111 09:02:13.343923 (+ 5us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:02:13.344027 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER)
0111 09:02:13.344032 (+ 5us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:13.344034 (+ 2us) write_op.cc:454] Released schema lock
0111 09:02:13.344036 (+ 2us) write_op.cc:335] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":32,"txn-participant-registration.run_cpu_time_us":281,"txn-participant-registration.run_wall_time_us":5971,"child_traces":[["op",{"apply.queue_time_us":43,"prepare.queue_time_us":104,"prepare.run_cpu_time_us":137,"prepare.run_wall_time_us":279,"replication_time_us":1052320,"child_traces":[["op",{}]]}]]}
W20250111 09:02:13.360787 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.361178 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.366456 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.368333 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.381680 19040 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 31 similar messages]
I20250111 09:02:13.610396 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:02:13.613018 17612 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:13.613166 17612 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 140 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:13.613242 17612 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 140 FOLLOWER]: Advancing to term 141
I20250111 09:02:13.614920 17612 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 141 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:13.615324 17612 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 141 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:13.615674 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 141 candidate_status { last_received { term: 140 index: 4252 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:13.615783 17476 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 140 LEADER]: Stepping down as leader of term 140
I20250111 09:02:13.615820 17476 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 140 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:13.615931 17476 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4253, Committed index: 4253, Last appended: 140.4254, Last appended by leader: 4254, Current term: 140, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:13.616016 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 140 FOLLOWER]: Advancing to term 141
W20250111 09:02:13.616350 19044 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.002s ago.
I20250111 09:02:13.616892 17476 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 141 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 141 because replica has last-logged OpId of term: 140 index: 4254, which is greater than that of the candidate, which has last-logged OpId of term: 140 index: 4252.
I20250111 09:02:13.617415 17612 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:13.617583 17612 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 140 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:13.617691 17612 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 140 FOLLOWER]: Advancing to term 141
I20250111 09:02:13.617882 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 141 candidate_status { last_received { term: 140 index: 4252 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
W20250111 09:02:13.618428 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:13.618755 17614 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 141 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 140. Current term is 141. Ops: [140.4253-140.4253]
I20250111 09:02:13.618824 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 141 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:13.619011 19051 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 141 FOLLOWER]: Leader election lost for term 141. Reason: could not achieve majority
I20250111 09:02:13.619709 17612 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 141 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:13.620294 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 141 candidate_status { last_received { term: 140 index: 4326 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:13.620380 17474 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 140 LEADER]: Stepping down as leader of term 140
I20250111 09:02:13.620376 17612 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 141 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:13.620414 17474 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 140 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:13.620514 17474 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4326, Committed index: 4326, Last appended: 140.4329, Last appended by leader: 4329, Current term: 140, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:13.620604 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 140 FOLLOWER]: Advancing to term 141
I20250111 09:02:13.620613 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 141 candidate_status { last_received { term: 140 index: 4326 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:13.621347 17474 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 141 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 141 because replica has last-logged OpId of term: 140 index: 4329, which is greater than that of the candidate, which has last-logged OpId of term: 140 index: 4326.
W20250111 09:02:13.621779 17455 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.621845 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:13.621583 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 141 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
W20250111 09:02:13.622020 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:13.622256 19018 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 141 FOLLOWER]: Leader election lost for term 141. Reason: could not achieve majority
W20250111 09:02:13.622319 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.623800 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.624228 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.625478 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.626029 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.627168 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.627573 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.628085 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.630925 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.632618 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.633376 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.633620 19056 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.634369 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.634883 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.635936 19056 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.636003 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.637429 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.638870 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.639166 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.640102 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.641141 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.641830 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.642370 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.643715 19056 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.644032 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.644738 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.645973 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.646955 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.648770 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.649140 19056 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.650377 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.650444 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.651337 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.654173 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.654878 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.655057 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.655349 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.656509 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.658816 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.659826 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.660887 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.661847 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.664208 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.665036 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.666550 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.666636 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.667121 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.668875 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.669824 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.672323 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.672973 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.675168 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.676677 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.677951 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.678984 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.679102 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.681420 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.682569 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.686271 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.687407 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.687974 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.691259 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.691927 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.692412 19058 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.692473 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.693702 19058 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.696830 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.699095 19058 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.700176 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.701735 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.704070 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.705947 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.706347 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.709147 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.709800 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.711520 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.715019 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.715728 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.715898 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.721460 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.721653 19058 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.723454 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.724779 19058 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.728053 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.728152 19058 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.729416 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.729481 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.738251 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.739346 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.739336 19056 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.739773 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.741115 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.743660 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.744537 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.745321 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.752898 19058 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.756141 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.756469 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.757220 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.759205 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.759828 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.760949 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.761126 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.771427 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.772856 19058 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.772924 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.774715 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.776613 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.776751 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.777316 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.778164 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.787421 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.791232 19056 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.791781 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.791826 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.794649 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.795250 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.796602 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.797291 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.806479 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.809906 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.811054 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.812388 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.813910 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.816040 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.817200 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.819200 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.824040 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.830196 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.831636 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.832743 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.836032 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.837456 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.837535 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.840947 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.842120 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.851924 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.852674 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.855818 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.858508 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.859345 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.859463 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.862108 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.863516 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.872718 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.873718 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.879290 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.880192 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.883589 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.883946 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.884057 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.884549 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.897814 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.898514 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.903682 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.905831 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.907033 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.907317 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.907959 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.910640 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.920967 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.925030 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.926110 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.930152 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.930509 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.930999 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.934330 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.935711 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.946990 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.952064 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.952572 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.954408 18911 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 50 similar messages]
W20250111 09:02:13.954456 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.956040 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.958293 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.960014 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.964203 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.972978 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.978137 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.978229 19058 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.980353 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.980427 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.983520 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:13.988569 19058 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.990012 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:13.998449 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.004685 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.004756 19056 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.007982 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.008129 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.012696 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.014781 19056 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.020318 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.028728 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.031006 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.033169 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.035329 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.037452 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.039624 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.044430 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.048406 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.060322 19058 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 44 similar messages]
W20250111 09:02:14.060389 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.060381 19058 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.064429 19058 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.064694 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.066849 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.067094 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.075574 19058 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.078049 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.088315 19056 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.089481 19056 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.092494 19056 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.095764 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.095865 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.098330 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.105823 19056 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.108259 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.118602 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.121021 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.123065 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.126425 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.129581 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.131731 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.136689 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.141587 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.151521 19058 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.152781 19058 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.153980 19058 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.158718 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.160847 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.161980 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.171087 19058 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.174446 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.186528 19056 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.186573 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.186690 19059 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.191748 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.197225 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.197438 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.202515 19059 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.206988 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.221601 18911 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.221613 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.222425 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.226228 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.230423 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.230542 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.237187 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.244462 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.255018 19058 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.256994 19058 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.257102 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.260636 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.266785 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.267918 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.270460 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.280476 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.291436 19057 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.291597 19059 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.292467 19056 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.297619 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.303133 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.306775 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.307014 19056 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.317622 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.329751 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.330998 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.331619 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.333901 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.338058 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.342941 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.346233 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.354024 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.367961 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.370113 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.371327 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.372254 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.375434 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.379643 19008 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.383610 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.395175 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.407397 19056 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 57 similar messages]
W20250111 09:02:14.407464 19056 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.408624 19056 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.409776 19056 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.411950 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.415589 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.418807 19056 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.422919 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.434944 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.447674 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.450062 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.450479 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.451579 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.457176 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:14.457751 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:02:14.458252 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:14.458364 17340 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 140 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:14.458438 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 140 FOLLOWER]: Advancing to term 141
I20250111 09:02:14.459302 17340 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 141 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:14.459653 17340 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 141 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:14.459915 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 141 candidate_status { last_received { term: 140 index: 4254 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:14.459985 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 141 candidate_status { last_received { term: 140 index: 4254 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:14.460111 17612 raft_consensus.cc:2388] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 141 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 in current term 141: Already voted for candidate b5fb421fa3354606aecbeafe55a0565c in this term.
I20250111 09:02:14.460196 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:14.460290 17340 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 140 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:14.460346 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 140 FOLLOWER]: Advancing to term 141
I20250111 09:02:14.460808 17474 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 141 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 141.
I20250111 09:02:14.460960 17340 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 141 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:14.461082 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 141 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:14.461263 17340 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 141 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:14.461510 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 141 candidate_status { last_received { term: 140 index: 4329 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:14.461539 19061 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 141 FOLLOWER]: Leader election won for term 141
I20250111 09:02:14.461676 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 141 candidate_status { last_received { term: 140 index: 4329 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:14.461799 17612 raft_consensus.cc:2388] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 141 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 in current term 141: Already voted for candidate b5fb421fa3354606aecbeafe55a0565c in this term.
I20250111 09:02:14.462021 19061 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 141 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:14.462374 17474 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 141 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 141.
I20250111 09:02:14.462368 19061 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4253, Committed index: 4253, Last appended: 140.4254, Last appended by leader: 4254, Current term: 141, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:14.462606 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 141 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters: b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:14.462800 19062 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 141 FOLLOWER]: Leader election won for term 141
I20250111 09:02:14.462935 19062 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 141 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:14.463172 19062 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4326, Committed index: 4326, Last appended: 140.4329, Last appended by leader: 4329, Current term: 141, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:14.464229 19055 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:14.464370 17474 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 141 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 140 index: 4254. Preceding OpId from leader: term: 141 index: 4256. (index mismatch)
I20250111 09:02:14.464680 19062 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4255, Last known committed idx: 4253, Time since last communication: 0.000s
I20250111 09:02:14.464752 17612 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 141 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 140 index: 4252. Preceding OpId from leader: term: 141 index: 4256. (index mismatch)
I20250111 09:02:14.464861 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 140 to 141, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 141 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:14.465086 19062 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4255, Last known committed idx: 4252, Time since last communication: 0.000s
I20250111 09:02:14.469416 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 140 to 141, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 141 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:02:14.472730 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:14.475051 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.484297 17418 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:14.487884 17612 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 141 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 140 index: 4326. Preceding OpId from leader: term: 141 index: 4331. (index mismatch)
I20250111 09:02:14.487946 17474 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 141 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 140 index: 4329. Preceding OpId from leader: term: 141 index: 4331. (index mismatch)
I20250111 09:02:14.488309 19061 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4330, Last known committed idx: 4326, Time since last communication: 0.000s
I20250111 09:02:14.488430 19063 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4330, Last known committed idx: 4326, Time since last communication: 0.000s
I20250111 09:02:14.492251 19064 mvcc.cc:204] Tried to move back new op lower bound from 7113056806859390976 to 7113056806762766336. Current Snapshot: MvccSnapshot[applied={T|T < 7113056803282538496}]
W20250111 09:02:14.494005 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.499392 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.505321 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.505623 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.507716 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:14.509377 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:15.013585 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:02:15.014171 17473 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:15.014755 17473 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 141 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:15.014828 17473 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 141 FOLLOWER]: Advancing to term 142
I20250111 09:02:15.015852 17473 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 142 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:15.016119 17467 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 142 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 141. Current term is 142. Ops: []
I20250111 09:02:15.016249 17473 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 142 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:15.016970 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 142 candidate_status { last_received { term: 141 index: 4386 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:15.017103 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 141 FOLLOWER]: Advancing to term 142
I20250111 09:02:15.017205 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 142 candidate_status { last_received { term: 141 index: 4386 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:15.019053 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 142 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 142.
I20250111 09:02:15.019289 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 142 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:15.019521 19102 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 142 FOLLOWER]: Leader election won for term 142
I20250111 09:02:15.019646 19102 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 142 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:15.019646 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:15.019819 19102 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4382, Committed index: 4382, Last appended: 141.4386, Last appended by leader: 4385, Current term: 142, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:15.019873 17467 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 141 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:15.020391 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 141 FOLLOWER]: Advancing to term 142
I20250111 09:02:15.021302 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 141 to 142, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 142 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:15.021896 17467 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 142 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:15.022060 17474 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 142 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 141. Current term is 142. Ops: [141.4465-141.4465]
I20250111 09:02:15.022364 19063 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: INVALID_TERM, Last received: 141.4386, Next index: 4387, Last known committed idx: 4382, Time since last communication: 0.000s
I20250111 09:02:15.023027 19061 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 141 LEADER]: Stepping down as leader of term 141
I20250111 09:02:15.023090 19061 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 141 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:15.023224 19061 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4386, Committed index: 4386, Last appended: 141.4389, Last appended by leader: 4389, Current term: 141, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:15.023533 19061 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 141 FOLLOWER]: Advancing to term 142
I20250111 09:02:15.024814 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 142 candidate_status { last_received { term: 141 index: 4464 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:15.024912 17336 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 141 LEADER]: Stepping down as leader of term 141
I20250111 09:02:15.024951 17336 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 141 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:15.025099 17336 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4464, Committed index: 4464, Last appended: 141.4465, Last appended by leader: 4465, Current term: 141, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:15.024952 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:15.026381 19061 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: INVALID_TERM, Last received: 141.4464, Next index: 4465, Last known committed idx: 4464, Time since last communication: 0.000s
I20250111 09:02:15.026832 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 141 FOLLOWER]: Advancing to term 142
I20250111 09:02:15.027133 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 142 candidate_status { last_received { term: 141 index: 4464 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:15.027240 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 141 FOLLOWER]: Advancing to term 142
I20250111 09:02:15.027444 17467 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 142 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:15.027796 17336 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 142 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 142 because replica has last-logged OpId of term: 141 index: 4465, which is greater than that of the candidate, which has last-logged OpId of term: 141 index: 4464.
I20250111 09:02:15.028102 17614 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 142 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 142 because replica has last-logged OpId of term: 141 index: 4465, which is greater than that of the candidate, which has last-logged OpId of term: 141 index: 4464.
W20250111 09:02:15.028183 19087 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: Replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 142 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
I20250111 09:02:15.028561 17612 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 142 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 141. Current term is 142. Ops: [141.4379-141.4386]
I20250111 09:02:15.029219 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 142 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:15.029546 19102 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 142 FOLLOWER]: Leader election lost for term 142. Reason: could not achieve majority
W20250111 09:02:15.029690 19092 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.030041 19095 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.033285 19108 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 30 similar messages]
W20250111 09:02:15.033372 19108 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.035179 19108 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.037699 19108 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.038901 19109 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.042518 19109 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.043854 19109 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.048082 19095 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.048163 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.052142 19095 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.055955 19108 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.056087 19107 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.062168 19107 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.063701 19109 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.064963 19109 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.070919 19109 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.075790 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.077941 19095 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 48 similar messages]
W20250111 09:02:15.078011 19095 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.081221 19095 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.087086 19107 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.089262 19107 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.090266 19108 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.100292 19109 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.101239 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.104733 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.112597 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.116922 19095 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.117290 19092 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.127491 19108 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.131846 19108 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.132135 19107 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.141722 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.145291 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.148010 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.156584 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.161868 19092 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.167199 19092 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.175992 19107 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.181121 19107 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.186437 19107 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.193076 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.198083 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.204788 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.212467 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.216455 19092 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.224638 19092 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.232184 19107 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.235493 19107 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.244616 19107 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.253304 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.254374 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.266120 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.276643 19092 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.277931 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.291167 19092 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.299054 19107 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.301177 19107 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.312075 19107 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.321732 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.322897 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.336517 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.346076 19092 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.349519 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.364269 19092 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.374033 19107 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.374070 19108 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.389319 19108 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.400090 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.401582 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.414057 19110 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 26 similar messages]
W20250111 09:02:15.414101 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.429816 19092 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.432044 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:15.432828 17340 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 142 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 141 index: 4389. Preceding OpId from leader: term: 142 index: 4387. (term mismatch)
I20250111 09:02:15.432876 17340 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 4386
I20250111 09:02:15.432898 17340 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 141.4387
W20250111 09:02:15.432981 17340 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Aborted: Op aborted by new leader
I20250111 09:02:15.433225 17340 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 141.4388
W20250111 09:02:15.433344 17340 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Aborted: Op aborted by new leader
I20250111 09:02:15.433545 17340 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted WRITE_OP operation due to leader change: 141.4389
W20250111 09:02:15.433645 17340 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Aborted: Op aborted by new leader
I20250111 09:02:15.434109 19102 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4387, Last known committed idx: 4386, Time since last communication: 0.000s
I20250111 09:02:15.436569 17612 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 142 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 141 index: 4378. Preceding OpId from leader: term: 142 index: 4387. (index mismatch)
I20250111 09:02:15.436961 19111 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4387, Last known committed idx: 4376, Time since last communication: 0.000s
W20250111 09:02:15.444422 19092 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.452093 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.452180 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.454322 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.459741 19092 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.460135 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.463851 19095 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.465304 19108 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.466058 19107 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.468742 19107 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.468850 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.469147 19113 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.473280 19113 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.474295 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.476258 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.476969 19113 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.477798 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.480396 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.482126 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.483292 19095 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.487480 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.487481 19095 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.490257 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.491974 19113 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.494482 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.496434 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.496869 19113 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.498831 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.498927 19109 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.500205 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.501539 19109 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.504976 19109 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.507591 19109 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.507680 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.509042 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:15.509860 19095 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.511608 19113 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.514127 19113 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:15.515827 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:02:15.516322 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:15.516438 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 142 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:15.516502 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 142 FOLLOWER]: Advancing to term 143
W20250111 09:02:15.516709 19095 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:15.517445 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 143 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:15.518047 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 143 candidate_status { last_received { term: 142 index: 4404 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:15.518100 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 143 candidate_status { last_received { term: 142 index: 4404 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:15.518265 17474 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 142 LEADER]: Stepping down as leader of term 142
I20250111 09:02:15.518314 17474 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 142 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:15.518510 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 142 FOLLOWER]: Advancing to term 143
I20250111 09:02:15.518493 17474 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4404, Committed index: 4404, Last appended: 142.4404, Last appended by leader: 4404, Current term: 142, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:15.518659 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 142 FOLLOWER]: Advancing to term 143
I20250111 09:02:15.519694 17474 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 143 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 143.
I20250111 09:02:15.519796 17336 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 143 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 143.
I20250111 09:02:15.519867 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 143 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:15.519976 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 143 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:02:15.520159 19089 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 143 FOLLOWER]: Leader election won for term 143
I20250111 09:02:15.520237 19089 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 143 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:15.520340 19089 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4404, Committed index: 4404, Last appended: 142.4404, Last appended by leader: 4404, Current term: 143, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:15.520768 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:15.520869 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 142 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:15.520917 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 142 FOLLOWER]: Advancing to term 143
I20250111 09:02:15.521579 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 142 to 143, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 143 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:15.521731 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 143 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:15.521984 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:15.522418 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 143 candidate_status { last_received { term: 141 index: 4465 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:15.522502 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 142 FOLLOWER]: Advancing to term 143
I20250111 09:02:15.522454 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 143 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:15.522862 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 143 candidate_status { last_received { term: 141 index: 4465 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:15.523000 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 142 FOLLOWER]: Advancing to term 143
I20250111 09:02:15.523959 17336 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 143 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 143.
W20250111 09:02:15.524132 19113 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:15.524281 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 143 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
W20250111 09:02:15.524044 19095 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:15.524432 19060 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 143 FOLLOWER]: Leader election won for term 143
I20250111 09:02:15.524521 19060 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 143 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:15.524631 19060 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4464, Committed index: 4464, Last appended: 141.4465, Last appended by leader: 4465, Current term: 143, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:15.525405 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:15.525429 17474 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 143 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 143.
I20250111 09:02:15.526559 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 141 to 143, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 143 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
W20250111 09:02:15.527566 19113 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.529759 19095 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:15.532356 17474 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 143 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 141 index: 4464. Preceding OpId from leader: term: 143 index: 4467. (index mismatch)
I20250111 09:02:15.532389 17336 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 143 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 141 index: 4465. Preceding OpId from leader: term: 143 index: 4467. (index mismatch)
I20250111 09:02:15.532661 19060 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4466, Last known committed idx: 4464, Time since last communication: 0.000s
I20250111 09:02:15.532748 19089 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4466, Last known committed idx: 4464, Time since last communication: 0.000s
W20250111 09:02:15.536810 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.536902 19074 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.541131 19113 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.542214 19108 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.542259 19107 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.547185 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.549448 19107 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.551249 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.551775 19095 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.552311 19092 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:15.555755 17474 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 143 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 142 index: 4404. Preceding OpId from leader: term: 143 index: 4406. (index mismatch)
I20250111 09:02:15.556483 17340 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 143 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 142 index: 4404. Preceding OpId from leader: term: 143 index: 4406. (index mismatch)
I20250111 09:02:15.556532 19089 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4405, Last known committed idx: 4404, Time since last communication: 0.000s
I20250111 09:02:15.557094 19089 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4405, Last known committed idx: 4404, Time since last communication: 0.000s
W20250111 09:02:15.589862 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.592383 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.593698 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:15.596031 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:16.104426 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:02:16.104933 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:16.113384 17338 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 143 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:16.113452 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 143 FOLLOWER]: Advancing to term 144
I20250111 09:02:16.114334 17338 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 144 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:16.114836 17338 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 144 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:16.115772 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 144 candidate_status { last_received { term: 143 index: 4552 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:16.115875 17614 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 143 LEADER]: Stepping down as leader of term 143
I20250111 09:02:16.115917 17614 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 143 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:16.116024 17614 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4553, Committed index: 4553, Last appended: 143.4555, Last appended by leader: 4555, Current term: 143, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:16.116057 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 144 candidate_status { last_received { term: 143 index: 4552 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:16.116139 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 143 FOLLOWER]: Advancing to term 144
W20250111 09:02:16.116429 19070 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.009s ago.
I20250111 09:02:16.116834 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 143 FOLLOWER]: Advancing to term 144
I20250111 09:02:16.117797 17614 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 144 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 144 because replica has last-logged OpId of term: 143 index: 4555, which is greater than that of the candidate, which has last-logged OpId of term: 143 index: 4552.
I20250111 09:02:16.118108 17337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:16.118208 17337 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 143 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:16.118278 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 143 FOLLOWER]: Advancing to term 144
I20250111 09:02:16.119176 17337 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 144 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:16.119436 17337 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 144 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
W20250111 09:02:16.120241 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:16.120728 17477 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 144 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 144 because replica has last-logged OpId of term: 143 index: 4553, which is greater than that of the candidate, which has last-logged OpId of term: 143 index: 4552.
I20250111 09:02:16.120747 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 144 candidate_status { last_received { term: 143 index: 4618 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:16.120841 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 143 FOLLOWER]: Advancing to term 144
I20250111 09:02:16.120955 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 144 candidate_status { last_received { term: 143 index: 4618 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:16.121085 17614 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 143 LEADER]: Stepping down as leader of term 143
I20250111 09:02:16.121064 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 144 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:16.121121 17614 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 143 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:16.121234 17614 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4619, Committed index: 4619, Last appended: 143.4619, Last appended by leader: 4619, Current term: 143, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:16.121352 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 143 FOLLOWER]: Advancing to term 144
I20250111 09:02:16.121687 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 144 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 144 because replica has last-logged OpId of term: 143 index: 4619, which is greater than that of the candidate, which has last-logged OpId of term: 143 index: 4618.
I20250111 09:02:16.121776 17336 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 144 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 143. Current term is 144. Ops: [143.4619-143.4619]
I20250111 09:02:16.122200 17614 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 144 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 144 because replica has last-logged OpId of term: 143 index: 4619, which is greater than that of the candidate, which has last-logged OpId of term: 143 index: 4618.
I20250111 09:02:16.122464 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 144 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:16.123103 19127 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 144 FOLLOWER]: Leader election lost for term 144. Reason: could not achieve majority
I20250111 09:02:16.123203 19127 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 144 FOLLOWER]: Leader election lost for term 144. Reason: could not achieve majority
W20250111 09:02:16.123445 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.124056 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.124208 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.127732 19128 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 33 similar messages]
W20250111 09:02:16.127820 19128 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.129145 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.130713 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.130905 19128 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.131906 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.132644 19130 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 42 similar messages]
W20250111 09:02:16.132690 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.135340 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.136629 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.136752 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.142140 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.142306 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.145368 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.146504 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.147886 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.152192 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.153302 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.154884 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.157894 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.163058 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.163506 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.166778 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.167153 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.173640 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.175674 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.178332 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.179636 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.184469 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.185770 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.192691 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.193061 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.196555 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.197739 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.204794 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.206426 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.208396 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.211838 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.218650 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.219528 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.224735 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.224793 19128 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.233986 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.235493 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.238724 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.241779 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.247741 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.254313 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.256415 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.257396 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.264585 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.271112 19128 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.272292 19128 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.274363 19128 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.284282 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.287909 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.291137 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.292330 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.303337 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.305819 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.310380 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.312402 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.323081 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.327035 19128 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.329353 19128 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.331224 19128 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.345618 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.347759 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.348942 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.350170 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.369565 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.369829 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.371714 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.372604 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.390507 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.391669 19128 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.395859 19128 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.397898 19128 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.413923 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.417188 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.422384 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.423529 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.436654 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.442052 19123 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 75 similar messages]
W20250111 09:02:16.442108 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.448454 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.451772 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.460381 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.468538 19128 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.475793 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.475852 19128 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:16.483752 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:02:16.484194 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:16.484284 17467 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 144 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:16.484328 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 144 FOLLOWER]: Advancing to term 145
I20250111 09:02:16.485217 17467 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 145 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:16.485653 17467 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 145 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:16.485908 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 145 candidate_status { last_received { term: 143 index: 4553 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:16.485954 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 145 candidate_status { last_received { term: 143 index: 4553 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:16.486016 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 144 FOLLOWER]: Advancing to term 145
I20250111 09:02:16.486060 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 144 FOLLOWER]: Advancing to term 145
I20250111 09:02:16.486322 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:16.486433 17467 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 144 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:16.486479 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 144 FOLLOWER]: Advancing to term 145
I20250111 09:02:16.486876 17340 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 145 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 145.
I20250111 09:02:16.487079 17614 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 145 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 145 because replica has last-logged OpId of term: 143 index: 4555, which is greater than that of the candidate, which has last-logged OpId of term: 143 index: 4553.
I20250111 09:02:16.487172 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 145 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:16.487323 17467 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 145 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:16.487422 19132 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 145 FOLLOWER]: Leader election won for term 145
I20250111 09:02:16.487569 19132 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 145 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:16.487618 17467 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 145 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:16.487879 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 145 candidate_status { last_received { term: 143 index: 4619 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:16.487879 19132 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4550, Committed index: 4550, Last appended: 143.4553, Last appended by leader: 4553, Current term: 145, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:16.487964 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 144 FOLLOWER]: Advancing to term 145
I20250111 09:02:16.487946 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 145 candidate_status { last_received { term: 143 index: 4619 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:16.488046 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 144 FOLLOWER]: Advancing to term 145
I20250111 09:02:16.488718 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 145 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 145.
I20250111 09:02:16.488824 17340 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 145 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 145.
I20250111 09:02:16.488973 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 145 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:02:16.489145 19132 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 145 FOLLOWER]: Leader election won for term 145
I20250111 09:02:16.489265 19132 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 145 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:16.489378 19132 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4619, Committed index: 4619, Last appended: 143.4619, Last appended by leader: 4619, Current term: 145, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:16.489674 17181 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 143 to 145, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 145 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:02:16.491097 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:16.492607 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 143 to 145, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 145 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:16.498126 17614 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 145 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 143 index: 4555. Preceding OpId from leader: term: 145 index: 4555. (term mismatch)
I20250111 09:02:16.498190 17614 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 4554
I20250111 09:02:16.498194 17340 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 145 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 143 index: 4552. Preceding OpId from leader: term: 145 index: 4555. (index mismatch)
I20250111 09:02:16.498220 17614 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 143.4555
W20250111 09:02:16.498306 17614 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:02:16.498584 19132 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4554, Last known committed idx: 4547, Time since last communication: 0.000s
I20250111 09:02:16.498875 19132 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4554, Last known committed idx: 4553, Time since last communication: 0.000s
I20250111 09:02:16.498998 19132 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 143.4554 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4553
I20250111 09:02:16.500172 17614 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 4553
I20250111 09:02:16.500221 17614 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 143.4554
W20250111 09:02:16.500301 17614 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:02:16.503809 17340 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 145 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 143 index: 4618. Preceding OpId from leader: term: 145 index: 4621. (index mismatch)
I20250111 09:02:16.504179 19137 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4620, Last known committed idx: 4618, Time since last communication: 0.000s
I20250111 09:02:16.504208 17612 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 145 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 143 index: 4619. Preceding OpId from leader: term: 145 index: 4621. (index mismatch)
I20250111 09:02:16.504885 19132 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4620, Last known committed idx: 4619, Time since last communication: 0.000s
W20250111 09:02:16.507196 19072 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.521238 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.533205 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.536042 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.540608 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:16.715135 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:02:16.715633 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:16.715806 17614 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 145 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:16.716125 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 145 FOLLOWER]: Advancing to term 146
I20250111 09:02:16.719291 17614 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 146 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:16.719683 17614 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 146 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:16.720342 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 146 candidate_status { last_received { term: 145 index: 4606 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:16.720314 17612 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:16.720513 17612 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 145 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:16.720568 17612 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 145 FOLLOWER]: Advancing to term 146
I20250111 09:02:16.721345 17612 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 146 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:16.721560 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 146 candidate_status { last_received { term: 145 index: 4606 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:16.721629 17612 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 146 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:16.721673 17467 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 145 LEADER]: Stepping down as leader of term 145
I20250111 09:02:16.721719 17467 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 145 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:16.721992 17467 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4606, Committed index: 4606, Last appended: 145.4607, Last appended by leader: 4607, Current term: 145, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:16.722237 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 146 candidate_status { last_received { term: 145 index: 4666 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:16.722335 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 145 FOLLOWER]: Advancing to term 146
I20250111 09:02:16.723263 17340 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 146 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 146.
I20250111 09:02:16.723625 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 146 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:16.724319 19112 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 146 FOLLOWER]: Leader election won for term 146
I20250111 09:02:16.724408 19112 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 146 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:16.724517 19112 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4666, Committed index: 4666, Last appended: 145.4666, Last appended by leader: 4666, Current term: 146, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:16.725368 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 146 candidate_status { last_received { term: 145 index: 4666 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:16.726461 17340 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 146 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 145. Current term is 146. Ops: [145.4667-145.4667]
I20250111 09:02:16.726675 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 145 to 146, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 146 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:16.727422 19132 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: INVALID_TERM, Last received: 145.4666, Next index: 4667, Last known committed idx: 4666, Time since last communication: 0.000s
I20250111 09:02:16.727607 19137 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 145 LEADER]: Stepping down as leader of term 145
I20250111 09:02:16.727672 19137 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 145 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:16.727816 19137 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4666, Committed index: 4666, Last appended: 145.4669, Last appended by leader: 4669, Current term: 145, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:16.727963 19137 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 145 FOLLOWER]: Advancing to term 146
W20250111 09:02:16.729883 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:16.732050 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 145 FOLLOWER]: Advancing to term 146
I20250111 09:02:16.732853 17467 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 146 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 146 because replica has last-logged OpId of term: 145 index: 4607, which is greater than that of the candidate, which has last-logged OpId of term: 145 index: 4606.
I20250111 09:02:16.733214 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 146 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:16.734397 19124 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 146 FOLLOWER]: Leader election lost for term 146. Reason: could not achieve majority
W20250111 09:02:16.734860 19080 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.735796 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.735901 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:16.735980 17614 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 146 LEADER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 145. Current term is 146. Ops: [145.4667-145.4667]
W20250111 09:02:16.736212 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.738979 19128 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.741091 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.741856 19128 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.741856 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.742509 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.743966 19128 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.745194 19128 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.746413 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.746836 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.747370 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:16.748585 17336 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 146 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 145 index: 4666. Preceding OpId from leader: term: 146 index: 4668. (index mismatch)
I20250111 09:02:16.748656 17476 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 146 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 145 index: 4669. Preceding OpId from leader: term: 146 index: 4668. (term mismatch)
I20250111 09:02:16.748709 17476 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 4667
I20250111 09:02:16.748744 17476 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 145.4668
W20250111 09:02:16.748840 17476 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
I20250111 09:02:16.748890 19124 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4667, Last known committed idx: 4666, Time since last communication: 0.000s
I20250111 09:02:16.749048 17476 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 145.4669
W20250111 09:02:16.749150 17476 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
I20250111 09:02:16.749586 19124 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4667, Last known committed idx: 4666, Time since last communication: 0.000s
I20250111 09:02:16.749665 19124 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer 262623ac6df745c785ce267ea39bd265 log is divergent from this leader: its last log entry 145.4667 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4666
I20250111 09:02:16.750221 17476 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 4666
I20250111 09:02:16.750273 17476 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 145.4667
W20250111 09:02:16.750367 17476 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
I20250111 09:02:16.751259 19099 mvcc.cc:204] Tried to move back new op lower bound from 7113056816119422976 to 7113056816025423872. Current Snapshot: MvccSnapshot[applied={T|T < 7113056816119422976}]
W20250111 09:02:16.754132 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.755403 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.756438 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.757388 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.762598 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.763334 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.765412 19128 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.766014 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.768390 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.769476 19128 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.769974 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.773111 19147 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.773716 19128 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.774336 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.774509 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.775264 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.775310 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.776105 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.776870 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.778213 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.779759 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.780544 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.780992 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.781760 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.783169 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.784456 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.785336 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.786552 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.787133 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.789616 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.792256 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.792973 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.793258 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.794780 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.795243 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.795984 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.799846 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.800066 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.800100 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.802748 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.802803 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.804169 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.805868 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.807780 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.809170 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.810245 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.811851 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.812772 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.814370 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.817082 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.818408 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.820453 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.821336 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.822180 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.822211 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.823963 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.825204 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.830209 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.830930 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.832823 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.835870 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.837503 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.837898 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.838397 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.840346 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.844875 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.845971 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.846408 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.849220 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.849614 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.853215 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.853425 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.853948 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.856426 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.860158 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.860977 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.863083 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.863381 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.866688 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.868041 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.870599 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.870780 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.875790 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.878125 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.878163 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.879594 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.880286 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.883782 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.886036 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.887185 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.892822 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.892928 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.893718 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.896389 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.897374 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.899988 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.902105 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.904904 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.910094 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.911542 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.911983 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.914072 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.915519 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.918455 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.918869 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.923786 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.926923 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.928792 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.931869 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.933032 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.935489 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.939457 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.939607 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.940091 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.945585 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.950184 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.951558 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.952607 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.957515 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.959389 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.960724 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.960803 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.964569 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.969564 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.970594 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.971704 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.977459 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.977934 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.981469 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.984840 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.985180 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.990489 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:16.992782 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.993994 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:16.997480 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.001107 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.006706 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.007836 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.009526 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.014195 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.015620 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.018349 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.018350 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.025933 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.031831 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.032613 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.034204 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.038337 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.040613 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.040879 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.042815 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.049132 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.058146 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.058182 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.058835 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.062738 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.063508 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.065188 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.067768 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.073874 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.083647 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.083716 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.086649 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.087977 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.088361 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.089946 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.094123 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.100546 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.109930 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.111425 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.112639 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.114025 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.114095 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.116190 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.120715 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.125322 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.134836 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.138631 19129 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 75 similar messages]
W20250111 09:02:17.138695 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.139839 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.139863 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.141887 19131 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 66 similar messages]
W20250111 09:02:17.141930 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.144224 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.146597 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.152671 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.161993 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.168466 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.169430 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.170081 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.171553 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.173106 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.174115 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.184170 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.189050 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.196866 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.198066 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.199620 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.199775 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.200901 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.206194 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.213526 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.219115 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.225549 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.226831 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.228901 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.229750 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.230708 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.240053 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.248687 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.250396 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:17.252979 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:02:17.253444 17336 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:17.253573 17336 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 145 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:17.253655 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 145 FOLLOWER]: Advancing to term 146
I20250111 09:02:17.254535 17336 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 146 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:17.254802 17336 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 146 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:17.255106 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 146 candidate_status { last_received { term: 145 index: 4606 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:17.255159 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 146 candidate_status { last_received { term: 145 index: 4606 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:17.255259 17477 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 146 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 146 because replica has last-logged OpId of term: 145 index: 4607, which is greater than that of the candidate, which has last-logged OpId of term: 145 index: 4606.
I20250111 09:02:17.255298 17614 raft_consensus.cc:2388] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 146 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 in current term 146: Already voted for candidate b5fb421fa3354606aecbeafe55a0565c in this term.
I20250111 09:02:17.255612 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 146 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:17.255757 17336 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:17.255851 19149 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 146 FOLLOWER]: Leader election lost for term 146. Reason: could not achieve majority
I20250111 09:02:17.255858 17336 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 146 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:17.255928 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 146 FOLLOWER]: Advancing to term 147
I20250111 09:02:17.256733 17336 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 147 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:17.257310 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:17.257632 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 147 candidate_status { last_received { term: 146 index: 4674 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:17.257721 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 146 FOLLOWER]: Advancing to term 147
I20250111 09:02:17.257769 17336 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 147 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:17.258061 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 147 candidate_status { last_received { term: 146 index: 4674 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:17.258188 17614 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 146 LEADER]: Stepping down as leader of term 146
I20250111 09:02:17.258229 17614 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 146 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:17.258356 17614 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4674, Committed index: 4674, Last appended: 146.4674, Last appended by leader: 4674, Current term: 146, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:17.258522 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 146 FOLLOWER]: Advancing to term 147
I20250111 09:02:17.258708 17477 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 147 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 147.
I20250111 09:02:17.258932 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 147 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:17.259078 19149 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 147 FOLLOWER]: Leader election won for term 147
I20250111 09:02:17.259215 19149 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 147 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:17.259361 19149 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4674, Committed index: 4674, Last appended: 146.4674, Last appended by leader: 4674, Current term: 147, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:17.259547 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 147 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 147.
W20250111 09:02:17.260766 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:17.260864 17181 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 146 to 147, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 147 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:02:17.262054 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.263422 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.263989 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.274044 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.282625 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.282716 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.290333 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.291478 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.292104 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.295665 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.298382 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.309593 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.312973 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.315702 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.322466 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.325141 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.327339 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.327395 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.333372 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.346129 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.346161 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.350172 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.359169 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.359344 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.359190 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.360374 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.368041 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.381556 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.382896 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.383474 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.393950 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.394181 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.395222 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.396668 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.403784 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.418485 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.418584 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.422631 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.430475 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.432075 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.432425 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.432466 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.442984 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.452672 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.452731 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.458513 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.468564 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.469534 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.469834 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.471343 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.482674 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.489073 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.491796 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.496362 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.507691 19129 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.507694 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.507767 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.511412 19123 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 89 similar messages]
W20250111 09:02:17.511459 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.524044 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.527880 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.527987 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.534032 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.546561 19123 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.547483 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.547557 19155 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.552433 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.562904 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.566918 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.566968 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.574016 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.586699 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.587399 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.588304 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.592967 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.603397 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.610500 19130 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.610605 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.613953 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.627938 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.627979 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.631745 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.632985 19155 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:17.644394 17477 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 147 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 146 index: 4674. Preceding OpId from leader: term: 147 index: 4675. (index mismatch)
I20250111 09:02:17.644789 19149 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4675, Last known committed idx: 4674, Time since last communication: 0.000s
I20250111 09:02:17.647045 17614 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 147 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 146 index: 4674. Preceding OpId from leader: term: 147 index: 4675. (index mismatch)
I20250111 09:02:17.647886 19149 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4675, Last known committed idx: 4674, Time since last communication: 0.000s
W20250111 09:02:17.648250 19155 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.650959 19155 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.653482 19155 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.656770 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.668262 19155 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.669198 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.676096 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.677701 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.690299 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.693163 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.697348 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.702518 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.712507 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.713497 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.721387 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.722394 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.736024 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.736055 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.740540 19131 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.746347 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.755205 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.759534 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.764560 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.768375 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.781193 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.782255 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.787086 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.790290 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.801493 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.802635 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:17.809065 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:02:17.809535 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:17.809641 17477 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 146 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:17.809705 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 146 FOLLOWER]: Advancing to term 147
I20250111 09:02:17.810583 17477 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 147 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:17.810811 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:17.810914 17477 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 147 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:17.811133 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 147 candidate_status { last_received { term: 145 index: 4607 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:17.811244 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 146 FOLLOWER]: Advancing to term 147
I20250111 09:02:17.811313 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 147 candidate_status { last_received { term: 145 index: 4607 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:17.811417 17614 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 146 FOLLOWER]: Advancing to term 147
I20250111 09:02:17.811960 17336 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 147 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 147.
I20250111 09:02:17.812042 17614 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 147 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 147.
I20250111 09:02:17.812199 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 147 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:17.812260 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:17.812361 17477 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 147 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:17.812410 19159 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 147 FOLLOWER]: Leader election won for term 147
I20250111 09:02:17.812418 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 147 FOLLOWER]: Advancing to term 148
I20250111 09:02:17.812498 19159 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 147 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:17.812637 19159 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4606, Committed index: 4606, Last appended: 145.4607, Last appended by leader: 4607, Current term: 147, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:17.813205 17477 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 148 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:17.813498 17477 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 148 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:17.813732 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 148 candidate_status { last_received { term: 147 index: 4675 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:17.813827 17336 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 147 LEADER]: Stepping down as leader of term 147
I20250111 09:02:17.813871 17336 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 147 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:17.813875 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 148 candidate_status { last_received { term: 147 index: 4675 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:17.813974 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 147 FOLLOWER]: Advancing to term 148
I20250111 09:02:17.813992 17336 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4675, Committed index: 4675, Last appended: 147.4675, Last appended by leader: 4675, Current term: 147, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:17.814149 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 147 FOLLOWER]: Advancing to term 148
I20250111 09:02:17.814199 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 145 to 147. New cstate: current_term: 147 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:17.814661 17614 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 148 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 148.
I20250111 09:02:17.814915 17336 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 148 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 148.
I20250111 09:02:17.814899 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 148 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:02:17.815131 19159 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 148 FOLLOWER]: Leader election won for term 148
I20250111 09:02:17.815225 19159 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 148 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:17.815651 19159 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4675, Committed index: 4675, Last appended: 147.4675, Last appended by leader: 4675, Current term: 148, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:17.816177 17614 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 147 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 145 index: 4606. Preceding OpId from leader: term: 147 index: 4609. (index mismatch)
I20250111 09:02:17.816229 17336 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 147 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 145 index: 4606. Preceding OpId from leader: term: 147 index: 4609. (index mismatch)
I20250111 09:02:17.816555 19159 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4608, Last known committed idx: 4606, Time since last communication: 0.000s
I20250111 09:02:17.816677 19164 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4608, Last known committed idx: 4605, Time since last communication: 0.000s
I20250111 09:02:17.818572 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 147 to 148, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 148 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:17.820348 19169 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50258 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=5959, attempt_no=0}) took 1111 ms. Trace:
I20250111 09:02:17.820401 19169 rpcz_store.cc:276] 0111 09:02:16.711662 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:16.711700 (+ 38us) service_pool.cc:225] Handling call
0111 09:02:17.820329 (+1108629us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:16.712317 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 2a4e17d0e2894eaa95b650f1af0ce650
0111 09:02:16.712366 (+ 49us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:16.712367 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:02:16.712368 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:16.712515 (+ 147us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:02:16.712548 (+ 33us) tablet.cc:698] Row locks acquired
0111 09:02:16.712549 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:02:16.712571 (+ 22us) write_op.cc:270] Start()
0111 09:02:16.712586 (+ 15us) write_op.cc:276] Timestamp: P: 1736586136712515 usec, L: 0
0111 09:02:16.712587 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:02:16.720061 (+ 7474us) log.cc:844] Serialized 307 byte log entry
0111 09:02:16.720313 (+ 252us) spinlock_profiling.cc:243] Waited 7.38 ms on lock 0x556630db2314. stack: 00007ff9585ec1af 00007ff9585ec7a1 00007ff95815489d 00005565fc536bee 00005565fc536c41 00005565fc53af7a 00007ff95a1899d2 00007ff95a67ecf3 00007ff95a67de58 00007ff95a67d726 00007ff95a6830b6 00005565fc5c3bbd 00007ff95860fa47 00007ff958610330 00007ff958611c7c 00005565fc5c3bbd
0111 09:02:16.731810 (+ 11497us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x556630f677d0. stack: 00007ff9585ec1af 00007ff9585ec7a1 00007ff95815489d 00005565fc536bee 00005565fc536c41 00007ff95af41909 00007ff95af41877 00007ff95a126c13 00007ff95a125fb9 00007ff95a12ac98 00005565fc5c3bbd 00007ff95860fa47 00007ff958610330 00007ff958611c7c 00005565fc5c3bbd 00007ff9585ff494
0111 09:02:17.819733 (+1087923us) op_driver.cc:464] REPLICATION: finished
0111 09:02:17.819898 (+ 165us) write_op.cc:301] APPLY: starting
0111 09:02:17.819914 (+ 16us) tablet.cc:1364] starting BulkCheckPresence
0111 09:02:17.820107 (+ 193us) tablet.cc:1367] finished BulkCheckPresence
0111 09:02:17.820109 (+ 2us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:02:17.820146 (+ 37us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:02:17.820173 (+ 27us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=96
0111 09:02:17.820176 (+ 3us) write_op.cc:312] APPLY: finished
0111 09:02:17.820217 (+ 41us) log.cc:844] Serialized 27 byte log entry
0111 09:02:17.820263 (+ 46us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:17.820271 (+ 8us) write_op.cc:454] Released schema lock
0111 09:02:17.820274 (+ 3us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":128,"num_ops":1,"prepare.queue_time_us":357,"prepare.run_cpu_time_us":476,"prepare.run_wall_time_us":8038,"raft.queue_time_us":21,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":11456,"replication_time_us":1107124,"spinlock_wait_cycles":43105664,"thread_start_us":77,"threads_started":1}]]}
W20250111 09:02:17.824564 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.827353 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.829869 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:17.830545 17614 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 148 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 147 index: 4675. Preceding OpId from leader: term: 148 index: 4677. (index mismatch)
W20250111 09:02:17.830889 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:17.831702 19159 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4676, Last known committed idx: 4675, Time since last communication: 0.000s
W20250111 09:02:17.833243 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.837903 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:17.841523 17336 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 148 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 147 index: 4675. Preceding OpId from leader: term: 148 index: 4677. (index mismatch)
I20250111 09:02:17.841878 19159 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4676, Last known committed idx: 4675, Time since last communication: 0.000s
W20250111 09:02:17.843030 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.866681 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.885917 17590 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.908493 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.913681 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:17.918210 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.918529 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:17.919592 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:18.013864 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:02:18.014331 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:18.023578 17611 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 147 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:18.023653 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 147 FOLLOWER]: Advancing to term 148
I20250111 09:02:18.024644 17611 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 148 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:18.025099 17611 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 148 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:18.025797 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:18.025890 17611 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 148 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:18.025942 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 148 FOLLOWER]: Advancing to term 149
I20250111 09:02:18.026712 17611 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 149 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:18.027019 17611 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 149 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:18.027673 17611 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 149 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 148. Current term is 149. Ops: [148.4718-148.4718]
I20250111 09:02:18.028335 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 149 candidate_status { last_received { term: 148 index: 4717 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:18.028443 17476 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 148 LEADER]: Stepping down as leader of term 148
I20250111 09:02:18.028483 17476 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 148 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:18.028620 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 148 candidate_status { last_received { term: 147 index: 4665 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:18.028609 17476 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4717, Committed index: 4717, Last appended: 148.4718, Last appended by leader: 4718, Current term: 148, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:18.028728 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 148 FOLLOWER]: Advancing to term 149
I20250111 09:02:18.028733 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 147 FOLLOWER]: Advancing to term 148
I20250111 09:02:18.029711 17476 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 149 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 149 because replica has last-logged OpId of term: 148 index: 4718, which is greater than that of the candidate, which has last-logged OpId of term: 148 index: 4717.
I20250111 09:02:18.029793 17337 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 148 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 148 because replica has last-logged OpId of term: 147 index: 4667, which is greater than that of the candidate, which has last-logged OpId of term: 147 index: 4665.
I20250111 09:02:18.029943 17340 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 148 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 147. Current term is 148. Ops: []
I20250111 09:02:18.030364 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 149 candidate_status { last_received { term: 148 index: 4717 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:18.030447 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 148 FOLLOWER]: Advancing to term 149
I20250111 09:02:18.030405 19188 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: INVALID_TERM, Last received: 147.4667, Next index: 4668, Last known committed idx: 4666, Time since last communication: 0.000s
I20250111 09:02:18.031399 19156 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 147 LEADER]: Stepping down as leader of term 147
I20250111 09:02:18.031446 19156 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 147 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
W20250111 09:02:18.031512 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:18.031569 19156 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4667, Committed index: 4667, Last appended: 147.4667, Last appended by leader: 4667, Current term: 147, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:18.031687 19156 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 147 FOLLOWER]: Advancing to term 148
I20250111 09:02:18.032311 17340 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 149 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 149 because replica has last-logged OpId of term: 148 index: 4718, which is greater than that of the candidate, which has last-logged OpId of term: 148 index: 4717.
I20250111 09:02:18.032752 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 148 candidate_status { last_received { term: 147 index: 4665 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:18.033944 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 149 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:18.034185 19178 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 149 FOLLOWER]: Leader election lost for term 149. Reason: could not achieve majority
W20250111 09:02:18.034695 19193 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.035396 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.033849 19158 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: Replica 262623ac6df745c785ce267ea39bd265 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 148 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
W20250111 09:02:18.038741 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.038810 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:18.038832 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 148 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:18.038971 19178 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 148 FOLLOWER]: Leader election lost for term 148. Reason: could not achieve majority
W20250111 09:02:18.039323 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.040016 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.040160 19158 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.012s ago.
W20250111 09:02:18.040477 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.041597 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.042356 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.043303 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.044466 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.045785 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.045781 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.047919 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.047967 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.048328 19155 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.048492 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.050704 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.053121 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.053155 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.053210 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.053530 19155 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.054867 19110 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.057283 19199 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.057397 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.057746 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.057755 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.057801 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.059320 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.060942 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.061244 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.062027 19193 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.062215 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.062646 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.063498 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.063670 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.065501 19193 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.067521 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.067622 19193 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.067634 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.068120 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.068909 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.069059 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.069325 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.071197 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.072304 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.072319 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.074045 19193 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.075131 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.077283 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.077391 19193 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.078392 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.079116 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.079427 19199 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.080483 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.080507 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.083230 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.083547 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.084717 19199 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.085109 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.085937 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.086256 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.088099 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.089565 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.090327 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.092118 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.092231 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.092893 19193 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.095564 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.096786 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.097297 19193 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.097326 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.097925 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.098974 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.099435 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.100509 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.101914 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.101958 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.104053 19193 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.104389 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.106057 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.107183 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.109549 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.110661 19193 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.110667 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.112021 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.112855 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.114187 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.115598 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.115660 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.118752 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.119858 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.119915 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.120908 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.122181 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.122226 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.122671 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.124679 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.124732 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.129130 19193 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.130455 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.130889 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.131781 19193 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.132578 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.133792 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.134073 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.134589 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.135826 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.138327 19193 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.139957 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.142850 19148 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 69 similar messages]
W20250111 09:02:18.142906 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.143733 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.144133 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.144302 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.146253 19193 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 55 similar messages]
W20250111 09:02:18.146353 19193 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.146854 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.147502 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.148211 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.150154 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.150213 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.156325 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.156710 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.159029 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.159078 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.161098 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.161151 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.162276 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.162786 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.165271 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.166891 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.166934 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.172722 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.172806 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.173560 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.174562 19193 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.176510 19193 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.177304 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.179105 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.180470 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.180591 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.181581 19193 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.185074 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.187989 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.187986 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.188923 19193 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.191593 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.194669 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.195883 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.196884 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.198022 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.200227 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.201174 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.202214 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.204047 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.207345 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.208299 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.211642 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.214700 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.215183 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.215212 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.215888 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.220400 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.220662 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.221714 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.223773 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.224556 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.225971 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.233470 19193 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.233536 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.234863 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.235647 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.235740 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.240614 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.240839 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.242365 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.244100 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.245342 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.247891 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.252519 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.252589 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.254786 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.256496 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.257994 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.258875 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.260113 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.262212 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.263830 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.263887 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.269574 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.272572 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.273788 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.274407 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.276185 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.281168 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.281253 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.281955 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.282548 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.285329 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.285392 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.293100 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.295179 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.297052 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.298781 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.300640 17420 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.300779 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.302971 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.302997 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.304102 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.306195 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.308378 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.315330 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.318282 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.320941 17589 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.321550 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.324707 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.325081 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.326076 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.327351 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.328130 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.328789 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.329463 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.341126 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.344386 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.344661 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.347373 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.349607 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.350606 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.350910 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.350919 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.352150 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.352337 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.354441 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.366086 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.369626 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.371418 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.371549 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.374291 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.374513 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.375180 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.376751 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.378239 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.378275 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.380048 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.395030 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.395095 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.397110 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.397950 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.400426 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.401412 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.403127 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.403436 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.405416 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.405639 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.408402 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.420377 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.424391 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.425766 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.426275 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.426787 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.428941 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.429024 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.431012 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.431109 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.433812 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.436082 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.451378 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.453377 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.454607 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.454730 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.455677 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.456866 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.458595 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.460120 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.460467 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.461711 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.467262 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.480122 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.481912 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.482200 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.484179 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.484558 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.485343 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.486773 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.490056 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.491206 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.491497 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.496229 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.511078 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.511102 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.512130 19200 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 80 similar messages]
W20250111 09:02:18.512188 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.514273 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.514364 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.515152 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.518373 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.518419 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.519539 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.523514 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.526729 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.540179 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.541368 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.544605 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.546619 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.547180 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.547806 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.547816 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.550052 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.551294 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.553092 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.556694 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.571305 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.571938 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.576591 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.578070 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.578123 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.578179 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.579418 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.581923 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.582562 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.587307 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.588750 19148 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.604405 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.604792 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.610545 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.611990 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.612511 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.613435 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.613780 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.614162 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.616196 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.623332 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.623364 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:18.636493 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:02:18.636941 17337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:18.637080 17337 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 148 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:18.637143 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 148 FOLLOWER]: Advancing to term 149
W20250111 09:02:18.637224 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.637295 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:18.638001 17337 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 149 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:18.638370 17337 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 149 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:18.638679 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 149 candidate_status { last_received { term: 147 index: 4667 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:18.638721 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 149 candidate_status { last_received { term: 147 index: 4667 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:18.638799 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 148 FOLLOWER]: Advancing to term 149
I20250111 09:02:18.638813 17612 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 148 FOLLOWER]: Advancing to term 149
I20250111 09:02:18.639750 17612 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 149 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 149.
I20250111 09:02:18.639750 17477 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 149 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 149.
I20250111 09:02:18.640024 17337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:18.640036 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 149 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:18.640141 17337 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 149 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:18.640208 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 149 FOLLOWER]: Advancing to term 150
I20250111 09:02:18.640331 19201 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 149 FOLLOWER]: Leader election won for term 149
I20250111 09:02:18.640470 19201 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 149 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:18.640599 19201 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4666, Committed index: 4666, Last appended: 147.4667, Last appended by leader: 4667, Current term: 149, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:18.640933 17337 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 150 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:18.641237 17337 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 150 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:18.641479 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 150 candidate_status { last_received { term: 148 index: 4718 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:18.641575 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 149 FOLLOWER]: Advancing to term 150
I20250111 09:02:18.641571 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 150 candidate_status { last_received { term: 148 index: 4718 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:18.641647 17612 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 149 FOLLOWER]: Advancing to term 150
I20250111 09:02:18.641829 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 147 to 149, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 149 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:18.642364 17612 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 150 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 150.
I20250111 09:02:18.642364 17477 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 150 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 150.
I20250111 09:02:18.642597 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 150 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:18.642819 19201 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 150 FOLLOWER]: Leader election won for term 150
I20250111 09:02:18.642942 19201 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 150 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:18.643075 19201 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4717, Committed index: 4717, Last appended: 148.4718, Last appended by leader: 4718, Current term: 150, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:18.645272 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 148 to 150, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 150 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:02:18.645465 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:18.647650 17477 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 150 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 148 index: 4718. Preceding OpId from leader: term: 150 index: 4720. (index mismatch)
I20250111 09:02:18.647739 17612 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 150 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 148 index: 4717. Preceding OpId from leader: term: 150 index: 4720. (index mismatch)
I20250111 09:02:18.648105 19201 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4719, Last known committed idx: 4717, Time since last communication: 0.000s
I20250111 09:02:18.648445 17612 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 149 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 147 index: 4665. Preceding OpId from leader: term: 149 index: 4669. (index mismatch)
I20250111 09:02:18.648468 17477 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 149 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 147 index: 4667. Preceding OpId from leader: term: 149 index: 4669. (index mismatch)
W20250111 09:02:18.648901 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:18.649616 19201 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4668, Last known committed idx: 4667, Time since last communication: 0.000s
I20250111 09:02:18.649675 19207 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4719, Last known committed idx: 4715, Time since last communication: 0.000s
I20250111 09:02:18.649825 19201 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4668, Last known committed idx: 4663, Time since last communication: 0.001s
W20250111 09:02:18.673709 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.700070 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:18.947559 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:02:18.948416 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:18.948540 17467 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 149 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:18.948601 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 149 FOLLOWER]: Advancing to term 150
I20250111 09:02:18.949679 17467 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 150 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:18.950201 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 150 candidate_status { last_received { term: 149 index: 4744 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:18.950292 17340 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 149 LEADER]: Stepping down as leader of term 149
I20250111 09:02:18.950333 17340 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 149 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:18.950455 17340 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4744, Committed index: 4744, Last appended: 149.4746, Last appended by leader: 4746, Current term: 149, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:18.950526 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 149 FOLLOWER]: Advancing to term 150
I20250111 09:02:18.951478 17340 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 150 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 150 because replica has last-logged OpId of term: 149 index: 4746, which is greater than that of the candidate, which has last-logged OpId of term: 149 index: 4744.
I20250111 09:02:18.952096 17467 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 150 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:18.952672 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 150 candidate_status { last_received { term: 149 index: 4744 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:18.952773 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 149 FOLLOWER]: Advancing to term 150
I20250111 09:02:18.953687 17611 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 150 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 150 because replica has last-logged OpId of term: 149 index: 4745, which is greater than that of the candidate, which has last-logged OpId of term: 149 index: 4744.
W20250111 09:02:18.955046 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:18.955067 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 150 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:18.955227 19210 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 150 FOLLOWER]: Leader election lost for term 150. Reason: could not achieve majority
I20250111 09:02:18.955457 17467 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 150 FOLLOWER]: Rejecting Update request from peer fb270e42d7c9400f8c05bb9565db1972 for earlier term 149. Current term is 150. Ops: [149.4745-149.4745]
I20250111 09:02:18.955669 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
W20250111 09:02:18.954762 19225 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.956188 19226 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.956997 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:18.957186 17477 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 150 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:18.957245 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 150 FOLLOWER]: Advancing to term 151
I20250111 09:02:18.958024 17477 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 151 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:18.958751 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 151 candidate_status { last_received { term: 150 index: 4792 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:18.959225 17340 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 150 LEADER]: Stepping down as leader of term 150
I20250111 09:02:18.959285 17340 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 150 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:18.959416 17340 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4794, Committed index: 4794, Last appended: 150.4796, Last appended by leader: 4796, Current term: 150, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:18.959523 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 150 FOLLOWER]: Advancing to term 151
W20250111 09:02:18.959792 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:18.960346 17340 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 151 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 151 because replica has last-logged OpId of term: 150 index: 4796, which is greater than that of the candidate, which has last-logged OpId of term: 150 index: 4792.
I20250111 09:02:18.961324 17477 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 151 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:18.961362 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 151 candidate_status { last_received { term: 150 index: 4792 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
W20250111 09:02:18.961423 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:18.961656 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 151 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:18.961796 19210 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 151 FOLLOWER]: Leader election lost for term 151. Reason: could not achieve majority
W20250111 09:02:18.962565 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.963675 19238 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.963716 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.965351 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.965976 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.967447 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.970269 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.970319 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.970767 19240 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.974377 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.975533 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.977346 17297 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.977421 17283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.978569 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.984629 19195 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.984692 19238 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.984787 19239 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.985121 19241 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.987531 19241 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:18.993225 19240 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.994860 19240 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.994884 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.996042 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:18.998664 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.003742 17283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.004506 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.005673 17283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.006114 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.010704 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.016683 19241 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.016932 19239 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.018852 19239 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.018852 19241 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.023207 19241 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.030052 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.030162 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.030210 19240 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.031673 19240 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.035079 19240 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.042523 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.043700 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.045260 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.046067 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.046650 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.056339 19241 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.057533 19239 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.058818 19239 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.060137 19239 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.061544 19239 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.071024 19240 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.072283 19240 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.072362 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.075096 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.076356 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.086103 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.088130 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.090310 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.092531 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.094619 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.104431 19239 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.107576 19239 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.107789 19241 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.108995 19241 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.110098 19241 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.123723 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.123818 19240 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.125259 19240 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.125476 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.126768 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.141491 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.143849 17283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.143849 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.144980 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.146157 19154 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 73 similar messages]
W20250111 09:02:19.146209 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.162204 19241 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 63 similar messages]
W20250111 09:02:19.162264 19241 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.163398 19241 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.166879 19239 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.167002 19241 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.167977 19238 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.182792 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.184151 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.186784 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.186964 19240 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.187870 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.204623 17283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.207907 17283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.209955 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.209991 17283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.211412 17283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.229455 19241 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.229535 19238 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.232486 19238 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.232554 19241 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.235471 19241 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.251355 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.256275 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.256480 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.256589 19240 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.259078 19240 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.278915 17283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.283159 17283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.283602 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.283733 17283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.286252 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.305084 19241 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.309502 19241 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.309610 19238 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.310456 19239 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.311578 19239 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.332556 19240 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.335103 19240 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.336436 19240 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.336508 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.338045 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.359848 17283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.361948 17283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.363169 17283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.364342 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.367689 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.388846 19239 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.389956 19238 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.390043 19241 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.391026 19238 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.399315 19238 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.418179 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.419338 19240 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.419922 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.420048 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.430817 19198 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:19.447737 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.448858 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.451871 19154 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.453276 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:19.458704 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:02:19.459219 17612 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:19.459347 17612 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 150 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:19.459425 17612 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 150 FOLLOWER]: Advancing to term 151
I20250111 09:02:19.460412 17612 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 151 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:19.460647 17296 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:19.460803 17612 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 151 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:19.461191 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 151 candidate_status { last_received { term: 149 index: 4745 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:19.461311 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 150 FOLLOWER]: Advancing to term 151
I20250111 09:02:19.461393 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 151 candidate_status { last_received { term: 149 index: 4745 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:19.461493 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 150 FOLLOWER]: Advancing to term 151
I20250111 09:02:19.462244 17476 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 151 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 151.
I20250111 09:02:19.462406 17340 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 151 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 151 because replica has last-logged OpId of term: 149 index: 4746, which is greater than that of the candidate, which has last-logged OpId of term: 149 index: 4745.
I20250111 09:02:19.462546 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 151 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:02:19.462661 17612 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:19.462710 19237 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 151 FOLLOWER]: Leader election won for term 151
I20250111 09:02:19.462754 17612 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 150 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:19.462805 17612 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 150 FOLLOWER]: Advancing to term 151
I20250111 09:02:19.462844 19237 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 151 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:19.462982 19237 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4744, Committed index: 4744, Last appended: 149.4745, Last appended by leader: 4745, Current term: 151, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:19.463521 17612 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 151 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:19.463800 17612 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 151 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:19.464066 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 151 candidate_status { last_received { term: 150 index: 4796 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:19.464201 17476 raft_consensus.cc:2388] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 151 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c in current term 151: Already voted for candidate 262623ac6df745c785ce267ea39bd265 in this term.
I20250111 09:02:19.464478 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 151 candidate_status { last_received { term: 150 index: 4796 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:19.464779 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 149 to 151, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 151 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:19.465389 17340 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 151 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 151.
I20250111 09:02:19.465649 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 151 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:02:19.465823 19242 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 151 FOLLOWER]: Leader election won for term 151
I20250111 09:02:19.465910 19242 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 151 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:19.466039 19242 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4794, Committed index: 4794, Last appended: 150.4796, Last appended by leader: 4796, Current term: 151, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:19.467921 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 150 to 151, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 151 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
W20250111 09:02:19.477820 19238 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.482002 19238 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.483052 19238 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.485190 19238 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.491561 19238 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:19.514724 17476 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 151 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 149 index: 4744. Preceding OpId from leader: term: 151 index: 4747. (index mismatch)
I20250111 09:02:19.514767 17340 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 151 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 149 index: 4746. Preceding OpId from leader: term: 151 index: 4747. (index mismatch)
I20250111 09:02:19.515180 19242 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4746, Last known committed idx: 4744, Time since last communication: 0.000s
I20250111 09:02:19.515272 19242 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer fb270e42d7c9400f8c05bb9565db1972 log is divergent from this leader: its last log entry 149.4746 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4744
I20250111 09:02:19.515334 19237 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4746, Last known committed idx: 4742, Time since last communication: 0.000s
I20250111 09:02:19.516391 17340 raft_consensus.cc:1212] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 151 FOLLOWER]: Deduplicated request from leader. Original: 149.4744->[149.4745-151.4747] Dedup: 149.4745->[151.4746-151.4747]
I20250111 09:02:19.516458 17340 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting all ops after (but not including) 4745
I20250111 09:02:19.516481 17340 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972: Aborting uncommitted WRITE_OP operation due to leader change: 149.4746
W20250111 09:02:19.516582 17340 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Aborted: Op aborted by new leader
I20250111 09:02:19.524544 19250 mvcc.cc:204] Tried to move back new op lower bound from 7113056827449139200 to 7113056827241472000. Current Snapshot: MvccSnapshot[applied={T|T < 7113056827449139200 or (T in {7113056827449139200})}]
I20250111 09:02:19.535593 17340 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 151 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 150 index: 4796. Preceding OpId from leader: term: 151 index: 4798. (index mismatch)
I20250111 09:02:19.535650 17476 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 151 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 150 index: 4792. Preceding OpId from leader: term: 151 index: 4798. (index mismatch)
I20250111 09:02:19.536479 19265 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4797, Last known committed idx: 4794, Time since last communication: 0.000s
I20250111 09:02:19.536737 19265 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4797, Last known committed idx: 4790, Time since last communication: 0.000s
W20250111 09:02:19.568032 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.568410 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.570525 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.571978 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.574362 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.574669 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:19.580018 19246 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 58 similar messages]
I20250111 09:02:20.080281 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:02:20.081110 17337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:20.081224 17337 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 151 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:20.081275 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 151 FOLLOWER]: Advancing to term 152
I20250111 09:02:20.087704 17337 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 152 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:20.088497 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 152 candidate_status { last_received { term: 151 index: 4879 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:20.088639 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 151 FOLLOWER]: Advancing to term 152
I20250111 09:02:20.089985 17336 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 152 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 151. Current term is 152. Ops: [151.4880-151.4883]
I20250111 09:02:20.090318 17476 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 152 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 152 because replica has last-logged OpId of term: 151 index: 4885, which is greater than that of the candidate, which has last-logged OpId of term: 151 index: 4879.
I20250111 09:02:20.090682 17467 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 152 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 151. Current term is 152. Ops: [151.4886-151.4886]
I20250111 09:02:20.090752 17337 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 152 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:20.091217 19257 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: INVALID_TERM, Last received: 151.4885, Next index: 4886, Last known committed idx: 4884, Time since last communication: 0.000s
I20250111 09:02:20.091414 17337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:20.091571 19264 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 151 LEADER]: Stepping down as leader of term 151
I20250111 09:02:20.091635 19264 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 151 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:20.091782 17337 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 151 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:20.091764 19264 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4879, Committed index: 4885, Last appended: 151.4887, Last appended by leader: 4887, Current term: 151, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:20.091862 19257 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: INVALID_TERM, Last received: 151.4879, Next index: 4880, Last known committed idx: 4878, Time since last communication: 0.000s
I20250111 09:02:20.091913 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 151 FOLLOWER]: Advancing to term 152
I20250111 09:02:20.092171 19264 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 151 FOLLOWER]: Advancing to term 152
I20250111 09:02:20.092823 17337 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 152 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:20.093178 17337 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 152 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:20.093533 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 152 candidate_status { last_received { term: 151 index: 4947 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:20.093611 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 151 FOLLOWER]: Advancing to term 152
I20250111 09:02:20.095144 17337 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 152 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 151. Current term is 152. Ops: [151.4948-151.4951]
I20250111 09:02:20.095327 17477 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 152 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 152 because replica has last-logged OpId of term: 151 index: 4951, which is greater than that of the candidate, which has last-logged OpId of term: 151 index: 4947.
I20250111 09:02:20.095834 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 152 candidate_status { last_received { term: 151 index: 4879 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:20.095912 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 152 candidate_status { last_received { term: 151 index: 4947 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:20.096015 17611 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 151 LEADER]: Stepping down as leader of term 151
I20250111 09:02:20.096055 17611 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 151 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:20.095952 17612 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 152 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 152 because replica has last-logged OpId of term: 151 index: 4887, which is greater than that of the candidate, which has last-logged OpId of term: 151 index: 4879.
I20250111 09:02:20.096168 17611 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4951, Committed index: 4951, Last appended: 151.4953, Last appended by leader: 4953, Current term: 151, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:20.096253 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 151 FOLLOWER]: Advancing to term 152
I20250111 09:02:20.096879 17611 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 152 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 152 because replica has last-logged OpId of term: 151 index: 4953, which is greater than that of the candidate, which has last-logged OpId of term: 151 index: 4947.
I20250111 09:02:20.097110 17467 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 152 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 151. Current term is 152. Ops: [151.4952-151.4952]
I20250111 09:02:20.098127 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 152 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:20.098376 19281 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 152 FOLLOWER]: Leader election lost for term 152. Reason: could not achieve majority
I20250111 09:02:20.098389 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 152 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:20.099216 19281 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 152 FOLLOWER]: Leader election lost for term 152. Reason: could not achieve majority
W20250111 09:02:20.102630 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.103377 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.104111 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.104529 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.106608 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.108582 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.108767 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.110845 19283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.112092 19283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.113971 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.116158 19283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.116180 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.117765 19283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.119117 19285 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.119342 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.120898 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.120901 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.121208 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.125674 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.126181 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.126181 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.126824 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.129724 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.130402 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.134658 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.135371 19282 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.135393 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.136045 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.136521 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.138738 19282 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.144408 19283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.144555 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.144780 19282 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.144814 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.145237 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.147349 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.152455 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.153064 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.154760 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.155689 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.157202 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.158980 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.164294 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.164502 19283 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 22 similar messages]
W20250111 09:02:20.164559 19283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.165288 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.166049 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.167232 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.167937 19283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.174126 19283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.176261 19283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.177608 19284 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 32 similar messages]
W20250111 09:02:20.177664 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.178675 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.180819 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.182039 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.187037 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.189230 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.191393 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.192767 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.192860 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.196581 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.198501 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.200698 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.203243 19283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.205662 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.208549 19283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.208844 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.212613 19283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.213704 19283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.217199 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.220265 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.222618 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.224651 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.228786 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.229941 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.236402 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.236572 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.238802 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.242758 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.247146 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.247997 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.251709 19283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.254544 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.255697 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.260929 19283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.262146 19282 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.265108 19282 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.268980 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.273310 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.274057 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.279099 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.280187 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.282171 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.287559 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.291049 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.293133 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.298873 19200 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.299835 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.303113 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.305820 19282 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.311933 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.315066 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.316848 19282 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.321036 19282 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.324132 19282 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.325501 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.331614 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.335954 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.338793 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.340795 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.344039 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.346284 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.353760 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.357923 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.362509 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.364554 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.368901 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.370280 19282 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.377740 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.378983 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.386910 19282 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.387987 19282 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.392114 19282 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.394883 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.403438 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.405900 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:20.411195 19285 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.411231 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.418085 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.418900 19279 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:20.422588 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:02:20.423035 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:20.423141 17476 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 152 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:20.423209 17476 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 152 FOLLOWER]: Advancing to term 153
I20250111 09:02:20.423982 17476 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 153 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:20.424306 17476 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 153 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:20.424607 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 153 candidate_status { last_received { term: 151 index: 4885 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:20.424607 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 153 candidate_status { last_received { term: 151 index: 4885 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:20.424718 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 152 FOLLOWER]: Advancing to term 153
I20250111 09:02:20.424719 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 152 FOLLOWER]: Advancing to term 153
I20250111 09:02:20.425135 17476 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:20.425267 17476 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 152 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:20.425331 17476 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 152 FOLLOWER]: Advancing to term 153
I20250111 09:02:20.425697 17611 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 153 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 153 because replica has last-logged OpId of term: 151 index: 4887, which is greater than that of the candidate, which has last-logged OpId of term: 151 index: 4885.
I20250111 09:02:20.425778 17336 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 153 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 153.
I20250111 09:02:20.426107 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 153 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters: b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:20.426152 17476 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 153 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:20.426400 19286 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 153 FOLLOWER]: Leader election won for term 153
I20250111 09:02:20.426508 17476 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 153 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:20.426522 19286 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 153 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:20.426803 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 153 candidate_status { last_received { term: 151 index: 4951 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:20.426890 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 152 FOLLOWER]: Advancing to term 153
I20250111 09:02:20.426883 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 153 candidate_status { last_received { term: 151 index: 4951 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:20.426872 19286 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4884, Committed index: 4884, Last appended: 151.4885, Last appended by leader: 4885, Current term: 153, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:20.426992 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 152 FOLLOWER]: Advancing to term 153
I20250111 09:02:20.427678 17611 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 153 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 153 because replica has last-logged OpId of term: 151 index: 4953, which is greater than that of the candidate, which has last-logged OpId of term: 151 index: 4951.
I20250111 09:02:20.427798 17336 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 153 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 153.
I20250111 09:02:20.428303 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 153 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters: b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:20.428519 19286 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 153 FOLLOWER]: Leader election won for term 153
W20250111 09:02:20.428565 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:20.428606 19286 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 153 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:20.428711 19286 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4951, Committed index: 4951, Last appended: 151.4951, Last appended by leader: 4951, Current term: 153, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:20.428738 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 151 to 153, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 153 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:02:20.431697 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:20.431640 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 151 to 153, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 153 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:20.439152 17611 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 153 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 151 index: 4887. Preceding OpId from leader: term: 153 index: 4887. (term mismatch)
I20250111 09:02:20.439211 17611 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 4886
I20250111 09:02:20.439237 17611 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 151.4887
W20250111 09:02:20.439330 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:02:20.439332 17336 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 153 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 151 index: 4879. Preceding OpId from leader: term: 153 index: 4887. (index mismatch)
I20250111 09:02:20.439621 17612 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 153 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 151 index: 4953. Preceding OpId from leader: term: 153 index: 4953. (term mismatch)
I20250111 09:02:20.439690 17612 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 4952
I20250111 09:02:20.439718 17612 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 151.4953
W20250111 09:02:20.439795 17612 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:02:20.439942 19287 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4886, Last known committed idx: 4878, Time since last communication: 0.000s
I20250111 09:02:20.440071 17336 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 153 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 151 index: 4947. Preceding OpId from leader: term: 153 index: 4953. (index mismatch)
I20250111 09:02:20.440368 19287 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4886, Last known committed idx: 4885, Time since last communication: 0.000s
I20250111 09:02:20.440461 19287 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 151.4886 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4885
I20250111 09:02:20.440605 19291 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4952, Last known committed idx: 4947, Time since last communication: 0.000s
I20250111 09:02:20.440750 19286 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4952, Last known committed idx: 4951, Time since last communication: 0.000s
I20250111 09:02:20.440829 19286 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 151.4952 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4951
I20250111 09:02:20.440954 17612 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 4885
I20250111 09:02:20.441028 17612 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 151.4886
W20250111 09:02:20.441111 17612 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:02:20.441280 17611 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 4951
I20250111 09:02:20.441324 17611 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 151.4952
W20250111 09:02:20.441390 17611 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
W20250111 09:02:20.449515 19282 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.449553 19283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.454710 19283 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:20.606600 19274 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 70 similar messages]
I20250111 09:02:21.125913 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:02:21.126629 17612 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:21.127882 17612 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 153 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:21.127956 17612 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 153 FOLLOWER]: Advancing to term 154
I20250111 09:02:21.129575 17612 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 154 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:21.130072 17612 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 154 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:21.130612 17340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 154 candidate_status { last_received { term: 153 index: 5031 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:21.130740 17340 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 153 FOLLOWER]: Advancing to term 154
I20250111 09:02:21.130813 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 154 candidate_status { last_received { term: 153 index: 5031 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:21.130936 17467 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 153 LEADER]: Stepping down as leader of term 153
I20250111 09:02:21.130986 17467 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 153 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:21.131104 17467 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5030, Committed index: 5030, Last appended: 153.5031, Last appended by leader: 5031, Current term: 153, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:21.131203 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 153 FOLLOWER]: Advancing to term 154
I20250111 09:02:21.132113 17467 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 154 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 154.
I20250111 09:02:21.132592 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 154 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:02:21.132719 19264 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 154 FOLLOWER]: Leader election won for term 154
I20250111 09:02:21.132772 19264 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 154 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:21.132850 17340 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 154 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 154.
I20250111 09:02:21.132858 19264 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5030, Committed index: 5030, Last appended: 153.5031, Last appended by leader: 5031, Current term: 154, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:21.133103 17614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
W20250111 09:02:21.133350 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:21.134507 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 153 to 154, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 154 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
W20250111 09:02:21.135020 19299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.135526 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.136690 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:21.137480 17614 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 153 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:21.137553 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 153 FOLLOWER]: Advancing to term 154
I20250111 09:02:21.138921 17614 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 154 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:21.140271 17467 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 154 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 153 index: 5031. Preceding OpId from leader: term: 154 index: 5033. (index mismatch)
I20250111 09:02:21.140362 17338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 154 candidate_status { last_received { term: 153 index: 5124 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:21.140625 19257 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5032, Last known committed idx: 5030, Time since last communication: 0.000s
I20250111 09:02:21.140861 17614 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 154 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:21.141383 17337 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 154 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 153 index: 5031. Preceding OpId from leader: term: 154 index: 5033. (index mismatch)
I20250111 09:02:21.141446 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 154 candidate_status { last_received { term: 153 index: 5124 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:21.141546 17477 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 153 LEADER]: Stepping down as leader of term 153
I20250111 09:02:21.141589 17477 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 153 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:21.141695 17477 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5124, Committed index: 5124, Last appended: 153.5126, Last appended by leader: 5126, Current term: 153, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:21.141798 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 153 FOLLOWER]: Advancing to term 154
I20250111 09:02:21.141820 19264 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5032, Last known committed idx: 5030, Time since last communication: 0.000s
I20250111 09:02:21.142647 17477 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 154 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 154 because replica has last-logged OpId of term: 153 index: 5126, which is greater than that of the candidate, which has last-logged OpId of term: 153 index: 5124.
I20250111 09:02:21.143069 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 154 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c; no voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:21.143429 19298 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 154 FOLLOWER]: Leader election lost for term 154. Reason: could not achieve majority
W20250111 09:02:21.146520 19248 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.146569 19305 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.148651 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.151302 19299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.157954 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.163475 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.165637 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.168222 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.168623 19304 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 25 similar messages]
W20250111 09:02:21.168690 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.175460 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.176316 19299 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.177204 19311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.179548 19284 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 73 similar messages]
W20250111 09:02:21.179611 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.179886 19285 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.182662 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.184041 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.184643 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.185091 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.186630 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.190192 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.190534 19285 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.190973 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.191298 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.194868 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.196986 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.197273 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.199169 19285 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.205830 19285 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.206153 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.206657 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.208611 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.209173 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.212860 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.216279 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.216604 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.220535 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.221143 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.222733 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.225106 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.227155 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.229214 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.233357 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.234548 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.236568 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.242508 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.242921 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.246212 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.246405 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.248852 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.253960 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.255993 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.258167 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.258342 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.263845 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.267004 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.268975 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.271044 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.274420 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.281553 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.282877 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.285574 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.286664 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.290314 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.296502 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.296924 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.298506 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.305930 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.309161 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.313686 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.313899 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.315683 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.323879 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.326474 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.331550 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.332067 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.333171 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.341451 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.345013 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.350370 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.351460 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.351596 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.359841 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.366513 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.369495 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.370388 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.372989 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.380604 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.385365 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.392217 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.392390 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.392623 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.403615 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.406489 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.412539 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.414425 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.415661 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.426874 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.431342 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.436278 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.439244 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.439301 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.453660 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.455250 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.457757 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.461216 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.461863 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.477131 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.481561 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.482090 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.485720 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.486860 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.505220 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.506850 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.508785 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.509513 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.511505 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.529228 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.535517 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.535674 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.538214 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.538960 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:21.540378 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:02:21.540766 17337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:21.540866 17337 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 154 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:21.540928 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 154 FOLLOWER]: Advancing to term 155
I20250111 09:02:21.541671 17337 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 155 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:21.541963 17337 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 155 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:21.542213 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 155 candidate_status { last_received { term: 154 index: 5040 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:21.542325 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 154 FOLLOWER]: Advancing to term 155
I20250111 09:02:21.542359 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 155 candidate_status { last_received { term: 154 index: 5040 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:21.542472 17611 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 154 LEADER]: Stepping down as leader of term 154
I20250111 09:02:21.542517 17611 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 154 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:21.542641 17611 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5040, Committed index: 5040, Last appended: 154.5040, Last appended by leader: 5040, Current term: 154, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:21.542773 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 154 FOLLOWER]: Advancing to term 155
I20250111 09:02:21.543251 17467 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 155 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 155.
I20250111 09:02:21.543514 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 155 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:21.543677 19306 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 155 FOLLOWER]: Leader election won for term 155
I20250111 09:02:21.543731 17611 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 155 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 155.
I20250111 09:02:21.543767 19306 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 155 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:21.543885 19306 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5040, Committed index: 5040, Last appended: 154.5040, Last appended by leader: 5040, Current term: 155, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:21.543982 17337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:21.544070 17337 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 153 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:21.544113 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 153 FOLLOWER]: Advancing to term 154
I20250111 09:02:21.544911 17337 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 154 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:21.545264 17337 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 154 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:21.545158 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 154 to 155, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 155 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:21.545521 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 154 candidate_status { last_received { term: 153 index: 5125 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:21.545670 17467 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 154 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 154 because replica has last-logged OpId of term: 153 index: 5126, which is greater than that of the candidate, which has last-logged OpId of term: 153 index: 5125.
I20250111 09:02:21.545712 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 154 candidate_status { last_received { term: 153 index: 5125 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:21.545841 17611 raft_consensus.cc:2388] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 154 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 in current term 154: Already voted for candidate b5fb421fa3354606aecbeafe55a0565c in this term.
I20250111 09:02:21.546449 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 154 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:21.546612 19307 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 154 FOLLOWER]: Leader election lost for term 154. Reason: could not achieve majority
W20250111 09:02:21.558151 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.564122 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.564718 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.565284 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.566118 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.588174 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.594121 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.594208 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.594861 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.596441 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.618140 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.622256 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.625552 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.626001 19310 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 34 similar messages]
W20250111 09:02:21.626056 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.627190 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.650246 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.654433 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.654461 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.659950 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.660027 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.684934 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.685024 19285 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.686120 19285 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.690822 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.691694 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.716150 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.718267 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.718315 19313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.721884 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.724912 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.751402 19310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.751441 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.752413 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.754678 19285 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.759873 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.784737 19285 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.785034 19312 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.787420 19312 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.788793 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.795663 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.819439 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.820396 19313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.822336 19313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.826000 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.835142 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.857165 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.857250 19314 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.860375 19314 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.865907 19312 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.873769 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.898062 19285 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.898288 19312 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.898375 19284 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.902071 19313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.909567 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.934337 19313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.935443 19313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.937552 19313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.938846 19314 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:21.950003 17611 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 155 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 154 index: 5040. Preceding OpId from leader: term: 155 index: 5041. (index mismatch)
I20250111 09:02:21.950343 19307 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5041, Last known committed idx: 5040, Time since last communication: 0.000s
W20250111 09:02:21.952207 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:21.952553 17467 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 155 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 154 index: 5040. Preceding OpId from leader: term: 155 index: 5041. (index mismatch)
I20250111 09:02:21.952864 19307 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5041, Last known committed idx: 5040, Time since last communication: 0.000s
W20250111 09:02:21.973563 19314 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.975535 19314 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.977752 19314 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:21.978919 19312 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:21.991221 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.013579 19312 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.015594 19312 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.020051 19312 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.020290 19313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.032238 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:22.056897 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.056919 19313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.062847 19314 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:22.062949 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:22.075074 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.097097 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:22.100229 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:22.104864 19313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.108250 19312 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.118876 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.141913 19312 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.143852 19312 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.147095 19312 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.150848 19313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.162830 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:22.185066 19313 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 51 similar messages]
W20250111 09:02:22.185118 19313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.187074 19313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.193404 19313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.196069 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:22.211318 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.229336 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:22.233657 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:22.236743 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:22.242206 19312 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 52 similar messages]
W20250111 09:02:22.242259 19312 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.256321 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.278090 19312 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.280164 19312 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.282188 19312 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.291968 19313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.305244 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:22.322993 19313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.325047 19313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.329361 19313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:22.334065 19317 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 154 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:02:22.334144 19317 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 154 FOLLOWER]: Advancing to term 155
I20250111 09:02:22.335021 19317 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 155 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:22.335342 19317 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 155 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:22.335592 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 155 candidate_status { last_received { term: 153 index: 5126 } } ignore_live_leader: false dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:22.335603 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 155 candidate_status { last_received { term: 153 index: 5126 } } ignore_live_leader: false dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:22.335919 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 155 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:22.336055 19317 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 155 FOLLOWER]: Leader election lost for term 155. Reason: could not achieve majority
W20250111 09:02:22.341219 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:22.352397 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.371398 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:22.372565 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:22.376742 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:22.381417 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:02:22.381886 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:22.381996 17467 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 155 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:22.382052 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 155 FOLLOWER]: Advancing to term 156
I20250111 09:02:22.382803 17467 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 156 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:22.383122 17467 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 156 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:22.383433 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 156 candidate_status { last_received { term: 155 index: 5041 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:22.383433 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 156 candidate_status { last_received { term: 155 index: 5041 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:22.383549 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 155 FOLLOWER]: Advancing to term 156
I20250111 09:02:22.383549 17337 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 155 LEADER]: Stepping down as leader of term 155
I20250111 09:02:22.383600 17337 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 155 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:22.383746 17337 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5041, Committed index: 5041, Last appended: 155.5041, Last appended by leader: 5041, Current term: 155, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:22.383905 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 155 FOLLOWER]: Advancing to term 156
I20250111 09:02:22.384573 17611 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 156 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 156.
I20250111 09:02:22.384872 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 156 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:02:22.384989 17337 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 156 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 156.
I20250111 09:02:22.385207 19317 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 156 FOLLOWER]: Leader election won for term 156
I20250111 09:02:22.385332 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:22.385397 19317 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 156 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:22.385432 17467 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 155 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:22.385478 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 155 FOLLOWER]: Advancing to term 156
I20250111 09:02:22.385535 19317 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5041, Committed index: 5041, Last appended: 155.5041, Last appended by leader: 5041, Current term: 156, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:22.386303 17467 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 156 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:22.386587 17467 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 156 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:22.386824 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 156 candidate_status { last_received { term: 153 index: 5126 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:22.386940 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 154 FOLLOWER]: Advancing to term 156
I20250111 09:02:22.386847 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 155 to 156, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 156 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:22.387082 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 156 candidate_status { last_received { term: 153 index: 5126 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:22.387192 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 154 FOLLOWER]: Advancing to term 156
I20250111 09:02:22.387674 17337 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 156 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 156.
I20250111 09:02:22.387904 17410 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 156 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:22.387948 17611 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 156 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 156.
I20250111 09:02:22.388442 19317 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 156 FOLLOWER]: Leader election won for term 156
I20250111 09:02:22.388523 19317 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 156 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:22.388690 19317 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5121, Committed index: 5121, Last appended: 153.5126, Last appended by leader: 5126, Current term: 156, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:22.390158 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 153 to 156. New cstate: current_term: 156 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:22.394062 17611 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 156 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 153 index: 5124. Preceding OpId from leader: term: 156 index: 5128. (index mismatch)
I20250111 09:02:22.394091 17337 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 156 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 153 index: 5125. Preceding OpId from leader: term: 156 index: 5128. (index mismatch)
I20250111 09:02:22.394481 19319 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5127, Last known committed idx: 5121, Time since last communication: 0.000s
I20250111 09:02:22.394757 19319 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5127, Last known committed idx: 5121, Time since last communication: 0.000s
I20250111 09:02:22.397305 19328 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50258 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=6532, attempt_no=0}) took 1273 ms. Trace:
I20250111 09:02:22.397408 19328 rpcz_store.cc:276] 0111 09:02:21.123927 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:21.123965 (+ 38us) service_pool.cc:225] Handling call
0111 09:02:22.397293 (+1273328us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:21.124482 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 6ab63c6543ad4a0ab83e76267a725edd
0111 09:02:21.124537 (+ 55us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:21.124539 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:02:21.124540 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:21.124574 (+ 34us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:02:21.124603 (+ 29us) tablet.cc:698] Row locks acquired
0111 09:02:21.124605 (+ 2us) write_op.cc:260] PREPARE: finished
0111 09:02:21.124627 (+ 22us) write_op.cc:270] Start()
0111 09:02:21.124645 (+ 18us) write_op.cc:276] Timestamp: P: 1736586141124593 usec, L: 0
0111 09:02:21.124647 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:02:21.124738 (+ 91us) log.cc:844] Serialized 307 byte log entry
0111 09:02:22.396405 (+1271667us) op_driver.cc:464] REPLICATION: finished
0111 09:02:22.396835 (+ 430us) write_op.cc:301] APPLY: starting
0111 09:02:22.396854 (+ 19us) tablet.cc:1364] starting BulkCheckPresence
0111 09:02:22.397067 (+ 213us) tablet.cc:1367] finished BulkCheckPresence
0111 09:02:22.397068 (+ 1us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:02:22.397105 (+ 37us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:02:22.397139 (+ 34us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=105
0111 09:02:22.397143 (+ 4us) write_op.cc:312] APPLY: finished
0111 09:02:22.397182 (+ 39us) log.cc:844] Serialized 27 byte log entry
0111 09:02:22.397221 (+ 39us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:22.397230 (+ 9us) write_op.cc:454] Released schema lock
0111 09:02:22.397234 (+ 4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":381,"num_ops":1,"prepare.queue_time_us":244,"prepare.run_cpu_time_us":294,"prepare.run_wall_time_us":295,"replication_time_us":1271735,"thread_start_us":63,"threads_started":1}]]}
I20250111 09:02:22.398283 19316 mvcc.cc:204] Tried to move back new op lower bound from 7113056839243497472 to 7113056839225954304. Current Snapshot: MvccSnapshot[applied={T|T < 7113056834046332928}]
I20250111 09:02:22.400043 19315 mvcc.cc:204] Tried to move back new op lower bound from 7113056839243497472 to 7113056839225954304. Current Snapshot: MvccSnapshot[applied={T|T < 7113056834046332928}]
I20250111 09:02:22.400550 19329 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50258 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=6531, attempt_no=0}) took 1276 ms. Trace:
I20250111 09:02:22.400624 19329 rpcz_store.cc:276] 0111 09:02:21.123876 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:21.124302 (+ 426us) service_pool.cc:225] Handling call
0111 09:02:22.400540 (+1276238us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:21.124796 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 6ab63c6543ad4a0ab83e76267a725edd
0111 09:02:21.124854 (+ 58us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:21.124856 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:02:21.124857 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:21.124896 (+ 39us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:02:21.124922 (+ 26us) tablet.cc:698] Row locks acquired
0111 09:02:21.124923 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:02:21.124945 (+ 22us) write_op.cc:270] Start()
0111 09:02:21.124962 (+ 17us) write_op.cc:276] Timestamp: P: 1736586141124911 usec, L: 0
0111 09:02:21.124964 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:02:21.125071 (+ 107us) log.cc:844] Serialized 307 byte log entry
0111 09:02:22.396526 (+1271455us) op_driver.cc:464] REPLICATION: finished
0111 09:02:22.396991 (+ 465us) write_op.cc:301] APPLY: starting
0111 09:02:22.397024 (+ 33us) tablet.cc:1364] starting BulkCheckPresence
0111 09:02:22.397599 (+ 575us) tablet.cc:1367] finished BulkCheckPresence
0111 09:02:22.397601 (+ 2us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:02:22.400320 (+ 2719us) spinlock_profiling.cc:243] Waited 2.51 ms on lock 0x556630adb50c. stack: 00007ff9585ec1af 00007ff9585ec7a1 00007ff95815489d 00005565fc536bee 00005565fc536c41 00005565fc53af7a 00005565fc57a415 00005565fc608d1c 00007ff95a5c798a 00007ff95a5c723d 00007ff95a69436a 00007ff95a680e43 00007ff95a680356 00007ff95a6832e7 00005565fc5c3bbd 00007ff95860fa47
0111 09:02:22.400360 (+ 40us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:02:22.400384 (+ 24us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=105
0111 09:02:22.400389 (+ 5us) write_op.cc:312] APPLY: finished
0111 09:02:22.400436 (+ 47us) log.cc:844] Serialized 27 byte log entry
0111 09:02:22.400476 (+ 40us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:22.400487 (+ 11us) write_op.cc:454] Released schema lock
0111 09:02:22.400492 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":430,"num_ops":1,"prepare.queue_time_us":272,"prepare.run_cpu_time_us":304,"prepare.run_wall_time_us":304,"replication_time_us":1271545,"spinlock_wait_cycles":5782784,"thread_start_us":76,"threads_started":1}]]}
I20250111 09:02:22.401748 19329 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.ParticipateInTransaction from 127.14.61.3:54455 (ReqId={client: c602c61032fa436092728430b0406df9, seq_no=4383, attempt_no=0}) took 1268 ms. Trace:
I20250111 09:02:22.401796 19329 rpcz_store.cc:276] 0111 09:02:21.133187 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:21.133532 (+ 345us) service_pool.cc:225] Handling call
0111 09:02:22.401737 (+1268205us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:21.134597 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:21.134622 (+ 25us) participant_op.cc:199] PREPARE: finished
0111 09:02:21.134637 (+ 15us) participant_op.cc:212] START. Timestamp: 1736586141134601
0111 09:02:21.134638 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:02:21.134702 (+ 64us) log.cc:844] Serialized 68 byte log entry
0111 09:02:22.396773 (+1262071us) op_driver.cc:464] REPLICATION: finished
0111 09:02:22.400920 (+ 4147us) participant_op.cc:265] APPLY: starting
0111 09:02:22.401001 (+ 81us) participant_op.cc:270] APPLY: finished
0111 09:02:22.401686 (+ 685us) log.cc:844] Serialized 16 byte log entry
0111 09:02:22.401713 (+ 27us) participant_op.cc:94] Released txn lock
0111 09:02:22.401724 (+ 11us) participant_op.cc:291] FINISH: Op applied
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4120,"prepare.queue_time_us":922,"prepare.run_cpu_time_us":137,"prepare.run_wall_time_us":137,"replication_time_us":1262118,"thread_start_us":67,"threads_started":1}]]}
I20250111 09:02:22.402127 19326 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50258 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=6533, attempt_no=0}) took 1265 ms. Trace:
I20250111 09:02:22.402172 19326 rpcz_store.cc:276] 0111 09:02:21.136369 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:21.136647 (+ 278us) service_pool.cc:225] Handling call
0111 09:02:22.402116 (+1265469us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:21.136879 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 6ab63c6543ad4a0ab83e76267a725edd
0111 09:02:21.136931 (+ 52us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:21.136932 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:02:21.136933 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:21.136969 (+ 36us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:02:21.136992 (+ 23us) tablet.cc:698] Row locks acquired
0111 09:02:21.136993 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:02:21.137038 (+ 45us) write_op.cc:270] Start()
0111 09:02:21.137055 (+ 17us) write_op.cc:276] Timestamp: P: 1736586141137002 usec, L: 0
0111 09:02:21.137057 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:02:21.137140 (+ 83us) log.cc:844] Serialized 307 byte log entry
0111 09:02:22.396891 (+1259751us) op_driver.cc:464] REPLICATION: finished
0111 09:02:22.401802 (+ 4911us) write_op.cc:301] APPLY: starting
0111 09:02:22.401814 (+ 12us) tablet.cc:1364] starting BulkCheckPresence
0111 09:02:22.401929 (+ 115us) tablet.cc:1367] finished BulkCheckPresence
0111 09:02:22.401930 (+ 1us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:02:22.401962 (+ 32us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:02:22.402001 (+ 39us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=105
0111 09:02:22.402005 (+ 4us) write_op.cc:312] APPLY: finished
0111 09:02:22.402039 (+ 34us) log.cc:844] Serialized 27 byte log entry
0111 09:02:22.402064 (+ 25us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:22.402070 (+ 6us) write_op.cc:454] Released schema lock
0111 09:02:22.402073 (+ 3us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4880,"num_ops":1,"prepare.queue_time_us":30,"prepare.run_cpu_time_us":296,"prepare.run_wall_time_us":296,"replication_time_us":1259818}]]}
W20250111 09:02:22.404254 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.408350 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:22.408650 19328 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50258 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=6529, attempt_no=0}) took 1293 ms. Trace:
I20250111 09:02:22.408706 19328 rpcz_store.cc:276] 0111 09:02:21.114790 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:21.115580 (+ 790us) service_pool.cc:225] Handling call
0111 09:02:22.408641 (+1293061us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:21.125120 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:21.125137 (+ 17us) participant_op.cc:199] PREPARE: finished
0111 09:02:21.125166 (+ 29us) participant_op.cc:212] START. Timestamp: 1736586141125129
0111 09:02:21.125168 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:02:21.125252 (+ 84us) log.cc:844] Serialized 68 byte log entry
0111 09:02:22.396651 (+1271399us) op_driver.cc:464] REPLICATION: finished
0111 09:02:22.397350 (+ 699us) participant_op.cc:265] APPLY: starting
0111 09:02:22.404324 (+ 6974us) participant_op.cc:270] APPLY: finished
0111 09:02:22.404344 (+ 20us) log.cc:844] Serialized 16 byte log entry
0111 09:02:22.404458 (+ 114us) participant_op.cc:94] Released txn lock
0111 09:02:22.404462 (+ 4us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:02:22.404696 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 6ab63c6543ad4a0ab83e76267a725edd
0111 09:02:22.404753 (+ 57us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:22.404754 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:02:22.404755 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:22.404796 (+ 41us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:02:22.404821 (+ 25us) tablet.cc:698] Row locks acquired
0111 09:02:22.404822 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:02:22.404846 (+ 24us) write_op.cc:270] Start()
0111 09:02:22.404865 (+ 19us) write_op.cc:276] Timestamp: P: 1736586142404781 usec, L: 0
0111 09:02:22.404868 (+ 3us) op_driver.cc:348] REPLICATION: starting
0111 09:02:22.404957 (+ 89us) log.cc:844] Serialized 307 byte log entry
0111 09:02:22.407528 (+ 2571us) op_driver.cc:464] REPLICATION: finished
0111 09:02:22.408242 (+ 714us) write_op.cc:301] APPLY: starting
0111 09:02:22.408255 (+ 13us) tablet.cc:1364] starting BulkCheckPresence
0111 09:02:22.408408 (+ 153us) tablet.cc:1367] finished BulkCheckPresence
0111 09:02:22.408409 (+ 1us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:02:22.408455 (+ 46us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:02:22.408481 (+ 26us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=105
0111 09:02:22.408485 (+ 4us) write_op.cc:312] APPLY: finished
0111 09:02:22.408523 (+ 38us) log.cc:844] Serialized 27 byte log entry
0111 09:02:22.408567 (+ 44us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:22.408573 (+ 6us) write_op.cc:454] Released schema lock
0111 09:02:22.408594 (+ 21us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"txn-participant-registration.queue_time_us":35,"txn-participant-registration.run_cpu_time_us":297,"txn-participant-registration.run_wall_time_us":9121,"child_traces":[["op",{"apply.queue_time_us":662,"apply.run_cpu_time_us":1154,"apply.run_wall_time_us":9284,"prepare.queue_time_us":258,"prepare.run_cpu_time_us":164,"prepare.run_wall_time_us":163,"replication_time_us":1271471,"thread_start_us":71,"threads_started":1,"child_traces":[["op",{"apply.queue_time_us":685,"num_ops":1,"prepare.queue_time_us":116,"prepare.run_cpu_time_us":314,"prepare.run_wall_time_us":314,"replication_time_us":2644}]]}]]}
W20250111 09:02:22.410899 19309 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:22.414618 19313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.415527 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.416766 19304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.417480 19314 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:22.418839 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.420221 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:22.420233 17611 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 156 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 155 index: 5041. Preceding OpId from leader: term: 156 index: 5043. (index mismatch)
I20250111 09:02:22.420579 17340 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 156 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 155 index: 5041. Preceding OpId from leader: term: 156 index: 5043. (index mismatch)
I20250111 09:02:22.421026 19345 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5042, Last known committed idx: 5041, Time since last communication: 0.000s
I20250111 09:02:22.421129 19319 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5042, Last known committed idx: 5041, Time since last communication: 0.000s
W20250111 09:02:22.425360 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.455482 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:22.482494 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:22.484983 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:22.955201 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:02:22.957175 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:22.967000 17611 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 156 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:22.967131 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 156 FOLLOWER]: Advancing to term 157
I20250111 09:02:22.968328 17611 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 157 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:22.969168 17611 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 157 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:22.969468 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 157 candidate_status { last_received { term: 156 index: 5179 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:22.969789 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:22.969897 17611 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 156 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:22.969950 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 156 FOLLOWER]: Advancing to term 157
I20250111 09:02:22.970659 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 157 candidate_status { last_received { term: 156 index: 5179 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:22.970770 17467 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 156 LEADER]: Stepping down as leader of term 156
I20250111 09:02:22.970813 17467 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 156 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:22.970922 17614 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 157 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 156. Current term is 157. Ops: [156.5180-156.5182]
I20250111 09:02:22.970932 17467 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5179, Committed index: 5179, Last appended: 156.5182, Last appended by leader: 5182, Current term: 156, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:22.971056 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 156 FOLLOWER]: Advancing to term 157
I20250111 09:02:22.971562 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 156 FOLLOWER]: Advancing to term 157
I20250111 09:02:22.972033 17467 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 157 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 157 because replica has last-logged OpId of term: 156 index: 5182, which is greater than that of the candidate, which has last-logged OpId of term: 156 index: 5179.
W20250111 09:02:22.974175 17432 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.974350 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:22.976609 17611 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 157 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:22.976926 17611 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 157 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:22.977177 17612 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 157 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 156. Current term is 157. Ops: []
W20250111 09:02:22.977711 19360 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:22.977749 19317 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: INVALID_TERM, Last received: 156.5265, Next index: 5266, Last known committed idx: 5264, Time since last communication: 0.000s
I20250111 09:02:22.977981 19317 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 156 LEADER]: Stepping down as leader of term 156
I20250111 09:02:22.978039 19317 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 156 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:22.978158 19317 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5265, Committed index: 5265, Last appended: 156.5267, Last appended by leader: 5267, Current term: 156, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:22.978201 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:22.978293 19317 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 156 FOLLOWER]: Advancing to term 157
I20250111 09:02:22.979120 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 157 candidate_status { last_received { term: 156 index: 5265 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:22.979228 17335 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 156 FOLLOWER]: Advancing to term 157
W20250111 09:02:22.979848 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:22.980480 17337 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 157 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 157.
I20250111 09:02:22.980726 17335 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 157 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 157.
I20250111 09:02:22.980757 17338 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 157 FOLLOWER]: Rejecting Update request from peer 262623ac6df745c785ce267ea39bd265 for earlier term 156. Current term is 157. Ops: [156.5179-156.5182]
I20250111 09:02:22.981043 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 157 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:22.981180 19352 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 157 FOLLOWER]: Leader election won for term 157
I20250111 09:02:22.981220 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 157 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:02:22.981361 19264 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 157 FOLLOWER]: Leader election won for term 157
I20250111 09:02:22.981436 19264 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 157 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:22.981564 19264 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5176, Committed index: 5176, Last appended: 156.5179, Last appended by leader: 5179, Current term: 157, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:22.982007 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:22.982198 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 157 candidate_status { last_received { term: 156 index: 5265 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:22.982348 17467 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 157 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 157 because replica has last-logged OpId of term: 156 index: 5267, which is greater than that of the candidate, which has last-logged OpId of term: 156 index: 5265.
I20250111 09:02:22.983738 19352 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 157 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:22.983871 19352 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5264, Committed index: 5264, Last appended: 156.5265, Last appended by leader: 5265, Current term: 157, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:22.983999 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:22.984931 17179 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 156 to 157, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 157 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:22.985087 17338 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 157 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 156 index: 5178. Preceding OpId from leader: term: 157 index: 5181. (index mismatch)
I20250111 09:02:22.985419 19352 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5180, Last known committed idx: 5176, Time since last communication: 0.000s
I20250111 09:02:22.986346 17467 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 157 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 156 index: 5182. Preceding OpId from leader: term: 157 index: 5181. (term mismatch)
I20250111 09:02:22.986407 17467 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 5180
I20250111 09:02:22.986440 17467 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 156.5181
W20250111 09:02:22.986537 17467 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
I20250111 09:02:22.986735 17467 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 156.5182
W20250111 09:02:22.986828 17467 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Aborted: Op aborted by new leader
I20250111 09:02:22.988648 19352 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5180, Last known committed idx: 5179, Time since last communication: 0.000s
I20250111 09:02:22.988759 19352 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer 262623ac6df745c785ce267ea39bd265 log is divergent from this leader: its last log entry 156.5180 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5179
W20250111 09:02:22.988936 19312 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.989317 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:22.989677 17477 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 5179
I20250111 09:02:22.989730 17477 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 156.5180
W20250111 09:02:22.989830 17477 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
I20250111 09:02:22.992251 17337 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 157 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 156 index: 5265. Preceding OpId from leader: term: 157 index: 5267. (index mismatch)
W20250111 09:02:22.992575 19349 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:22.993237 19346 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5266, Last known committed idx: 5264, Time since last communication: 0.000s
I20250111 09:02:22.994343 17477 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 157 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 156 index: 5267. Preceding OpId from leader: term: 157 index: 5267. (term mismatch)
I20250111 09:02:22.994397 17477 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 5266
I20250111 09:02:22.994421 17477 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted PARTICIPANT_OP operation due to leader change: 156.5267
W20250111 09:02:22.994485 17477 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Aborted: Op aborted by new leader
I20250111 09:02:22.995098 19340 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5266, Last known committed idx: 5265, Time since last communication: 0.000s
I20250111 09:02:22.995188 19340 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer 262623ac6df745c785ce267ea39bd265 log is divergent from this leader: its last log entry 156.5266 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5265
I20250111 09:02:22.995741 17477 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 5265
I20250111 09:02:22.995790 17477 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted WRITE_OP operation due to leader change: 156.5266
W20250111 09:02:22.995806 19360 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:22.995877 17477 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Aborted: Op aborted by new leader
I20250111 09:02:23.004351 17178 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 156 to 157, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 157 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
W20250111 09:02:23.005209 19360 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.036535 19347 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 32 similar messages]
I20250111 09:02:23.557677 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:02:23.558344 17335 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:23.560633 17335 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 157 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:23.560703 17335 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 157 FOLLOWER]: Advancing to term 158
I20250111 09:02:23.562467 17335 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 158 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:23.564994 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 158 candidate_status { last_received { term: 157 index: 5338 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:23.565130 17611 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 157 LEADER]: Stepping down as leader of term 157
I20250111 09:02:23.565210 17611 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 157 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:23.565382 17611 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5338, Committed index: 5338, Last appended: 157.5340, Last appended by leader: 5340, Current term: 157, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:23.565497 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 157 FOLLOWER]: Advancing to term 158
I20250111 09:02:23.566557 17611 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 158 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 158 because replica has last-logged OpId of term: 157 index: 5340, which is greater than that of the candidate, which has last-logged OpId of term: 157 index: 5338.
W20250111 09:02:23.566917 19353 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: Replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 158 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
I20250111 09:02:23.564180 17335 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 158 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:23.568496 17337 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 158 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 157. Current term is 158. Ops: []
I20250111 09:02:23.569175 17337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:23.569303 17337 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 157 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:23.569373 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 157 FOLLOWER]: Advancing to term 158
I20250111 09:02:23.570278 17337 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 158 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:23.570485 17335 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 158 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 157. Current term is 158. Ops: [157.5406-157.5408]
I20250111 09:02:23.570827 17473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 158 candidate_status { last_received { term: 157 index: 5405 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:23.571400 19352 consensus_queue.cc:1046] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: INVALID_TERM, Last received: 157.5405, Next index: 5406, Last known committed idx: 5404, Time since last communication: 0.000s
I20250111 09:02:23.571631 19352 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 157 LEADER]: Stepping down as leader of term 157
I20250111 09:02:23.571687 19352 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 157 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:23.571784 19352 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5405, Committed index: 5405, Last appended: 157.5408, Last appended by leader: 5408, Current term: 157, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:23.571894 19352 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 157 FOLLOWER]: Advancing to term 158
I20250111 09:02:23.574730 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 158 candidate_status { last_received { term: 157 index: 5338 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
W20250111 09:02:23.574896 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:23.575065 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 158 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:23.575362 19376 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 158 FOLLOWER]: Leader election lost for term 158. Reason: could not achieve majority
W20250111 09:02:23.575372 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.575685 19371 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:23.575807 17337 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 158 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:23.576134 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 158 candidate_status { last_received { term: 157 index: 5405 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:23.576269 17611 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 158 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 158 because replica has last-logged OpId of term: 157 index: 5408, which is greater than that of the candidate, which has last-logged OpId of term: 157 index: 5405.
I20250111 09:02:23.576579 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 158 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:23.576815 19376 raft_consensus.cc:2743] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 158 FOLLOWER]: Leader election lost for term 158. Reason: could not achieve majority
W20250111 09:02:23.577126 19375 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 12 similar messages]
W20250111 09:02:23.577173 19375 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.579564 19375 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.580893 19375 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.582391 19377 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 47 similar messages]
W20250111 09:02:23.582443 19377 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.583636 19377 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.585744 19377 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.587852 19375 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.588090 19377 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.589816 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.592672 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.595921 19371 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.596879 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.597699 19375 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.602672 19375 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.604678 19375 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.606582 19375 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.607821 19377 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.611418 19377 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.614003 19377 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.615083 19377 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.618623 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.623579 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.624109 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.625842 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.630733 19375 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.637195 19375 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.637825 19380 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.637825 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.644642 19377 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.648809 19377 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.650281 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.650295 19377 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.660142 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.660467 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.661119 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.665699 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.673607 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.674774 19380 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.677910 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.678210 19380 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.686942 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.690920 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.692255 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.695628 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.703217 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.707908 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.708896 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.711118 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.720099 19380 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.725541 19380 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.725638 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.727517 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.739598 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.742429 19377 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.742498 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.745595 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.758248 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.761171 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.761793 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.765167 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.777038 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.781064 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.783202 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.785321 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.796015 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.800539 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.806560 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.808739 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.817564 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.821782 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.830667 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.831311 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.843065 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.843087 19380 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.856194 19380 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.856258 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.866173 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.869133 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.878413 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.880515 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.890973 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.895648 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.904199 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.908485 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.917393 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.921408 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.930810 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.936131 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.943814 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.949126 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.958161 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.965595 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:23.972205 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.978819 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.985460 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:23.995918 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.003895 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.008694 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.015961 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.025239 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.036182 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:24.038130 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:02:24.038538 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:24.038630 17477 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 157 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:24.038691 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 157 FOLLOWER]: Advancing to term 158
I20250111 09:02:24.039489 17477 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 158 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:24.039808 17477 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 158 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:24.040108 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 158 candidate_status { last_received { term: 157 index: 5338 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:24.040122 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 158 candidate_status { last_received { term: 157 index: 5338 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:24.040246 17337 raft_consensus.cc:2388] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 158 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 in current term 158: Already voted for candidate fb270e42d7c9400f8c05bb9565db1972 in this term.
I20250111 09:02:24.040254 17611 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 158 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 158 because replica has last-logged OpId of term: 157 index: 5340, which is greater than that of the candidate, which has last-logged OpId of term: 157 index: 5338.
I20250111 09:02:24.040516 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:24.040534 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 158 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:24.040612 17477 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 157 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:24.040664 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 157 FOLLOWER]: Advancing to term 158
I20250111 09:02:24.040812 19381 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 158 FOLLOWER]: Leader election lost for term 158. Reason: could not achieve majority
W20250111 09:02:24.041435 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:24.041448 17477 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 158 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:24.041741 17477 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 158 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:24.041962 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 158 candidate_status { last_received { term: 157 index: 5408 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:24.042068 17337 raft_consensus.cc:2388] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 158 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 in current term 158: Already voted for candidate fb270e42d7c9400f8c05bb9565db1972 in this term.
I20250111 09:02:24.042066 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 158 candidate_status { last_received { term: 157 index: 5408 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:24.042778 17611 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 158 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 158.
I20250111 09:02:24.043004 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 158 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters: fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:24.043147 19381 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 158 FOLLOWER]: Leader election won for term 158
I20250111 09:02:24.043330 19381 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 158 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:24.043473 19381 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5404, Committed index: 5404, Last appended: 157.5408, Last appended by leader: 5408, Current term: 158, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:24.044631 17178 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 157 to 158, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 158 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:02:24.045622 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.056468 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.067947 19368 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 53 similar messages]
W20250111 09:02:24.067998 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.070600 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.077199 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.090626 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.100342 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.102280 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.107690 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.120837 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.131822 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.137432 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.140434 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.156256 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.163909 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.172161 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.173602 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.191190 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.200963 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.205078 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.207242 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.227469 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.239439 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.243453 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.245432 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.263917 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.279042 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.281684 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.283408 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.299844 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.315757 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.318707 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.321828 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.336030 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.353731 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.355724 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.363037 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.375139 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.394279 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.395160 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.403853 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.414332 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.435106 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.436149 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.444471 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.456526 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.478571 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.478634 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.486605 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.496544 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:24.497151 17611 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 158 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 157 index: 5408. Preceding OpId from leader: term: 158 index: 5409. (index mismatch)
I20250111 09:02:24.497491 19381 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5409, Last known committed idx: 5405, Time since last communication: 0.000s
I20250111 09:02:24.500608 17337 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 158 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 157 index: 5405. Preceding OpId from leader: term: 158 index: 5409. (index mismatch)
I20250111 09:02:24.501075 19381 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5409, Last known committed idx: 5404, Time since last communication: 0.000s
W20250111 09:02:24.502501 19396 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.513820 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.514117 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.517046 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.517127 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.516956 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.520610 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.521036 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.523020 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.523698 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.523728 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.526067 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.526226 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.532572 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.533797 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.533799 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.536947 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.536975 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.537418 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.542361 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.543507 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.543677 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.544296 19410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.546022 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.546871 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.551647 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.552798 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.553483 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.553663 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.554250 19410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.560145 19410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.560537 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.561374 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.563411 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.564635 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.565205 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.567675 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.568878 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.570077 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.570449 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.575098 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.575284 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.575419 19410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.580930 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.581125 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.581280 19379 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 58 similar messages]
W20250111 09:02:24.581342 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.585094 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.588126 19410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.588229 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.588317 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.590006 19388 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 58 similar messages]
W20250111 09:02:24.590052 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.594594 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.596031 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.597098 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.600522 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.601594 19410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.607381 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.607441 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.610072 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.611187 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.611244 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.615167 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.616130 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.618301 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.619838 19410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.623497 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.625864 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.630975 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.632644 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.634159 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.635627 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.635627 19410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.635977 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.643476 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.648308 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.648952 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.651101 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.651101 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.657173 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.658702 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.660727 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.662024 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.665642 19410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.665665 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.667083 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.674777 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.677232 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.681823 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.681994 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.684159 19410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.684319 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.684921 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.691345 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.695876 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.698662 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.700402 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.706460 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.706525 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.706578 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.711519 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.714781 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.718246 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.720932 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.724630 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.726667 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.731379 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.732475 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.733912 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.736426 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.742446 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.745682 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.754254 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.754295 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.756243 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.756322 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.759397 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.764638 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.766093 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.775883 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.776049 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.777374 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.780118 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.783448 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.784759 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.787810 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.798462 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.799700 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.801873 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.805694 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.806885 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.808688 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.809798 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.823489 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.824209 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.826711 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.827777 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.831014 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.832634 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.836182 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.846540 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.851975 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.852301 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.855487 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.855971 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.857554 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.860064 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.870584 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.872601 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.876951 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.877883 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.880455 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.884203 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.885264 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.895934 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.902236 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.907773 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.907930 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.908267 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.913308 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.913547 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.924081 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.924326 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.928104 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.936002 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.938074 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.943243 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.943905 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.955978 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.956033 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.960860 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.966773 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.967152 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.971457 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.974238 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.977655 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.978870 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.987692 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:24.997999 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:24.998322 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.001850 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.006425 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.010454 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:25.014019 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:02:25.014462 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:25.014560 17611 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 158 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:25.014623 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 158 FOLLOWER]: Advancing to term 159
W20250111 09:02:25.015043 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:25.015491 17611 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 159 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:25.015781 17611 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 159 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:25.016038 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 159 candidate_status { last_received { term: 157 index: 5340 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:25.016007 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 159 candidate_status { last_received { term: 157 index: 5340 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:25.016137 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 158 FOLLOWER]: Advancing to term 159
I20250111 09:02:25.016139 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 158 FOLLOWER]: Advancing to term 159
I20250111 09:02:25.016809 17337 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 159 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 159.
I20250111 09:02:25.016815 17477 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 159 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 159.
I20250111 09:02:25.017055 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:25.017056 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 159 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:25.017159 17611 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 158 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:25.017216 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 158 FOLLOWER]: Advancing to term 159
I20250111 09:02:25.017269 19408 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 159 FOLLOWER]: Leader election won for term 159
I20250111 09:02:25.017397 19408 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 159 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:25.017534 19408 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5338, Committed index: 5338, Last appended: 157.5340, Last appended by leader: 5340, Current term: 159, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:25.017906 17611 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 159 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:25.018180 17611 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 159 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:25.018464 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 159 candidate_status { last_received { term: 158 index: 5419 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:25.018468 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 159 candidate_status { last_received { term: 158 index: 5419 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:25.018571 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 158 FOLLOWER]: Advancing to term 159
I20250111 09:02:25.018574 17477 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 158 LEADER]: Stepping down as leader of term 158
I20250111 09:02:25.018613 17477 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 158 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:25.018734 17477 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5419, Committed index: 5419, Last appended: 158.5419, Last appended by leader: 5419, Current term: 158, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:25.018900 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 158 FOLLOWER]: Advancing to term 159
I20250111 09:02:25.018920 17178 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 157 to 159. New cstate: current_term: 159 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:25.019454 17337 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 159 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 159.
I20250111 09:02:25.019611 17477 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 159 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 159.
I20250111 09:02:25.019733 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 159 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:25.020288 19408 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 159 FOLLOWER]: Leader election won for term 159
I20250111 09:02:25.020370 19408 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 159 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:25.020478 19408 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5419, Committed index: 5419, Last appended: 158.5419, Last appended by leader: 5419, Current term: 159, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:25.022699 17178 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 158 to 159, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 159 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:25.023105 17477 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 159 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 157 index: 5338. Preceding OpId from leader: term: 159 index: 5342. (index mismatch)
I20250111 09:02:25.023429 19408 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5341, Last known committed idx: 5335, Time since last communication: 0.000s
I20250111 09:02:25.023468 17337 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 159 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 157 index: 5338. Preceding OpId from leader: term: 159 index: 5342. (index mismatch)
I20250111 09:02:25.023738 19408 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5341, Last known committed idx: 5334, Time since last communication: 0.000s
I20250111 09:02:25.026755 19395 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.ParticipateInTransaction from 127.14.61.3:36423 (ReqId={client: c602c61032fa436092728430b0406df9, seq_no=4610, attempt_no=0}) took 1469 ms. Trace:
I20250111 09:02:25.026829 19395 rpcz_store.cc:276] 0111 09:02:23.557228 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:23.557281 (+ 53us) service_pool.cc:225] Handling call
0111 09:02:25.026743 (+1469462us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:23.557591 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:23.557618 (+ 27us) participant_op.cc:199] PREPARE: finished
0111 09:02:23.557633 (+ 15us) participant_op.cc:212] START. Timestamp: 1736586143557587
0111 09:02:23.557634 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:02:23.559992 (+ 2358us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x558c4798e3b8. stack: 00007f3e831c61af 00007f3e831c67a1 00007f3e82d2e89d 0000558c2a807bee 0000558c2a807c41 0000558c2a80bf7a 00007f3e84d17a9a 00007f3e84d7ab25 00007f3e84d7a70b 00007f3e84d63862 00007f3e85258cf3 00007f3e85257e58 00007f3e85257726 00007f3e8525d0b6 0000558c2a894bbd 00007f3e831e9a47
0111 09:02:23.560057 (+ 65us) log.cc:844] Serialized 68 byte log entry
0111 09:02:25.026503 (+1466446us) op_driver.cc:464] REPLICATION: finished
0111 09:02:25.026588 (+ 85us) participant_op.cc:265] APPLY: starting
0111 09:02:25.026678 (+ 90us) participant_op.cc:270] APPLY: finished
0111 09:02:25.026704 (+ 26us) log.cc:844] Serialized 16 byte log entry
0111 09:02:25.026718 (+ 14us) participant_op.cc:94] Released txn lock
0111 09:02:25.026731 (+ 13us) participant_op.cc:291] FINISH: Op applied
Metrics: {"child_traces":[["op",{"apply.queue_time_us":26,"prepare.queue_time_us":153,"prepare.run_cpu_time_us":348,"prepare.run_wall_time_us":3583,"replication_time_us":1468850,"spinlock_wait_cycles":5115136}]]}
I20250111 09:02:25.027161 19396 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=6899, attempt_no=0}) took 1494 ms. Trace:
W20250111 09:02:25.027222 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:25.027213 19396 rpcz_store.cc:276] 0111 09:02:23.532730 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:23.532778 (+ 48us) service_pool.cc:225] Handling call
0111 09:02:25.027152 (+1494374us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:23.549459 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:23.549476 (+ 17us) participant_op.cc:199] PREPARE: finished
0111 09:02:23.549491 (+ 15us) participant_op.cc:212] START. Timestamp: 1736586143549446
0111 09:02:23.549492 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:02:23.549554 (+ 62us) log.cc:844] Serialized 68 byte log entry
0111 09:02:23.556819 (+ 7265us) op_driver.cc:464] REPLICATION: finished
0111 09:02:23.556889 (+ 70us) participant_op.cc:265] APPLY: starting
0111 09:02:23.560068 (+ 3179us) participant_op.cc:270] APPLY: finished
0111 09:02:23.560091 (+ 23us) log.cc:844] Serialized 16 byte log entry
0111 09:02:23.560109 (+ 18us) participant_op.cc:94] Released txn lock
0111 09:02:23.560114 (+ 5us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:02:23.561201 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 2a4e17d0e2894eaa95b650f1af0ce650
0111 09:02:23.561599 (+ 398us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:23.561601 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:02:23.561602 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:23.561644 (+ 42us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:02:23.561668 (+ 24us) tablet.cc:698] Row locks acquired
0111 09:02:23.561669 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:02:23.561689 (+ 20us) write_op.cc:270] Start()
0111 09:02:23.561703 (+ 14us) write_op.cc:276] Timestamp: P: 1736586143561647 usec, L: 0
0111 09:02:23.561704 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:02:23.561787 (+ 83us) log.cc:844] Serialized 307 byte log entry
0111 09:02:25.026546 (+1464759us) op_driver.cc:464] REPLICATION: finished
0111 09:02:25.026720 (+ 174us) write_op.cc:301] APPLY: starting
0111 09:02:25.026733 (+ 13us) tablet.cc:1364] starting BulkCheckPresence
0111 09:02:25.026921 (+ 188us) tablet.cc:1367] finished BulkCheckPresence
0111 09:02:25.026923 (+ 2us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:02:25.026993 (+ 70us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:02:25.027023 (+ 30us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=112
0111 09:02:25.027028 (+ 5us) write_op.cc:312] APPLY: finished
0111 09:02:25.027064 (+ 36us) log.cc:844] Serialized 27 byte log entry
0111 09:02:25.027093 (+ 29us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:25.027101 (+ 8us) write_op.cc:454] Released schema lock
0111 09:02:25.027105 (+ 4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"txn-participant-registration.queue_time_us":36,"txn-participant-registration.run_cpu_time_us":321,"txn-participant-registration.run_wall_time_us":15468,"child_traces":[["op",{"apply.queue_time_us":40,"apply.run_cpu_time_us":973,"apply.run_wall_time_us":3384,"prepare.queue_time_us":1074,"prepare.run_cpu_time_us":133,"prepare.run_wall_time_us":132,"replication_time_us":7307,"child_traces":[["op",{"apply.queue_time_us":153,"num_ops":1,"prepare.queue_time_us":977,"prepare.run_cpu_time_us":281,"prepare.run_wall_time_us":622,"replication_time_us":1464832}]]}]]}
W20250111 09:02:25.032269 17437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.032553 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.032639 19379 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.034627 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.035367 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.037070 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:25.044806 17477 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 159 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 158 index: 5419. Preceding OpId from leader: term: 159 index: 5421. (index mismatch)
I20250111 09:02:25.045246 17335 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 159 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 158 index: 5419. Preceding OpId from leader: term: 159 index: 5421. (index mismatch)
I20250111 09:02:25.045666 19404 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5420, Last known committed idx: 5419, Time since last communication: 0.000s
I20250111 09:02:25.045776 19408 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5420, Last known committed idx: 5419, Time since last communication: 0.000s
W20250111 09:02:25.058818 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.067824 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.068326 19421 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.070009 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.072329 19421 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.073614 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.073838 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.077930 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.079402 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.080163 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.080735 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.084255 19423 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 35 similar messages]
W20250111 09:02:25.087102 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.091006 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.093356 19378 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.094448 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.098363 19388 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.102129 19387 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.102805 19421 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.105472 19421 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:25.617132 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:02:25.618456 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:25.655977 17338 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 159 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:25.656095 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 159 FOLLOWER]: Advancing to term 160
I20250111 09:02:25.657213 17338 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 160 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:25.657999 17338 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 160 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:25.658253 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 160 candidate_status { last_received { term: 159 index: 5494 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:25.658677 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:25.658777 17338 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 159 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:25.658826 17338 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 159 FOLLOWER]: Advancing to term 160
I20250111 09:02:25.659631 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 160 candidate_status { last_received { term: 159 index: 5494 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:25.659762 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 159 FOLLOWER]: Advancing to term 160
I20250111 09:02:25.659803 17338 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 160 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:25.660207 17338 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 160 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:25.660459 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 160 candidate_status { last_received { term: 159 index: 5558 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:25.660552 17611 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 159 LEADER]: Stepping down as leader of term 159
I20250111 09:02:25.660589 17611 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 159 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:25.660686 17611 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5558, Committed index: 5558, Last appended: 159.5558, Last appended by leader: 5558, Current term: 159, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:25.660786 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 159 FOLLOWER]: Advancing to term 160
I20250111 09:02:25.661139 17340 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 160 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 159. Current term is 160. Ops: [159.5495-159.5506]
I20250111 09:02:25.661616 17611 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 160 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 160.
W20250111 09:02:25.661737 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:25.661818 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 160 candidate_status { last_received { term: 159 index: 5558 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:25.661907 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 159 FOLLOWER]: Advancing to term 160
I20250111 09:02:25.661931 19404 consensus_queue.cc:1046] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: INVALID_TERM, Last received: 159.5494, Next index: 5495, Last known committed idx: 5494, Time since last communication: 0.000s
I20250111 09:02:25.662093 19404 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 159 LEADER]: Stepping down as leader of term 159
I20250111 09:02:25.662137 19404 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 159 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:25.662174 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 160 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:25.662257 19404 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5504, Committed index: 5504, Last appended: 159.5507, Last appended by leader: 5507, Current term: 159, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:25.662396 19404 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 159 FOLLOWER]: Advancing to term 160
I20250111 09:02:25.663051 17474 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 160 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 160.
I20250111 09:02:25.664713 19439 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 160 FOLLOWER]: Leader election won for term 160
I20250111 09:02:25.664896 19439 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 160 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:25.665077 19439 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5558, Committed index: 5558, Last appended: 159.5558, Last appended by leader: 5558, Current term: 160, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:25.666864 17178 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 159 to 160, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 160 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:25.668488 17477 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 160 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 160 because replica has last-logged OpId of term: 159 index: 5504, which is greater than that of the candidate, which has last-logged OpId of term: 159 index: 5494.
I20250111 09:02:25.668697 17476 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 160 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 159. Current term is 160. Ops: [159.5505-159.5507]
I20250111 09:02:25.668743 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 160 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:25.668910 19439 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 160 FOLLOWER]: Leader election lost for term 160. Reason: could not achieve majority
W20250111 09:02:25.671561 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.672300 19437 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.672394 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.672430 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.672812 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:25.674593 17611 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 160 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 159 index: 5558. Preceding OpId from leader: term: 160 index: 5560. (index mismatch)
W20250111 09:02:25.674815 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:25.674899 17477 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 160 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 159 index: 5558. Preceding OpId from leader: term: 160 index: 5560. (index mismatch)
I20250111 09:02:25.675379 19439 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5559, Last known committed idx: 5558, Time since last communication: 0.000s
I20250111 09:02:25.675895 19439 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5559, Last known committed idx: 5558, Time since last communication: 0.000s
W20250111 09:02:25.680992 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.685396 19442 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 39 similar messages]
W20250111 09:02:25.685452 19442 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.688493 19447 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.689808 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.695148 19450 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 45 similar messages]
W20250111 09:02:25.695711 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.697937 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.699151 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.703452 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.703810 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.705762 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.706686 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.709156 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.710050 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.711120 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.712404 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.712981 17593 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.714291 19447 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.715999 19447 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.717247 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.719578 19447 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.720117 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.722162 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.722687 19446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.724496 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.725390 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.726760 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.726832 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.727169 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.727560 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.730705 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.730801 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.733301 19446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.735188 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.736102 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.737432 19446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.738134 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.739928 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.741029 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.741906 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.743291 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.744007 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.746517 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.747680 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.750325 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.750809 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.751741 19446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.754226 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.755020 19446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.756323 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.756446 19446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.761803 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.762066 19446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.762202 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.763911 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.765365 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.768440 19446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.772001 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.772853 19446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.772941 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.776132 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.776409 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.778223 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.782641 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.782829 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.786793 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.788015 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.788635 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.792591 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.793066 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.797490 19446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.798184 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.799682 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.800415 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.805227 19446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.806371 19446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.807886 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.810564 19446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.810642 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.812609 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.814117 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.819911 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.822609 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.824908 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.825296 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.826617 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.826709 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.828874 19446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.838244 19446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.840699 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.840704 19446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.841339 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.843375 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.843531 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.846585 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.856148 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.857568 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.859048 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.859125 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.859978 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.860863 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.865796 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.874657 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.874722 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.877533 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.879439 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.879593 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.879995 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.888510 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.895961 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.896265 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.896466 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.896926 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.899632 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.900746 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.908828 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.915942 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.918995 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.919412 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.919665 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.919776 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.923995 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.929771 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.937814 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.939919 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.942236 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.943099 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.943487 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.949306 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.951750 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.958940 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.963371 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.964849 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.964865 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.965194 19446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.973193 19446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.974486 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:25.980470 19446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.984651 19446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.987522 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.987725 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.988665 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.997447 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:25.999740 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.004930 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.009232 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.011983 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.012259 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.014565 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.025991 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.028136 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.032393 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.034485 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.038373 19446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.038532 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.038847 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.053576 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.056327 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.057667 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.061846 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.063335 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.064603 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.065799 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.082628 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.083906 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.085593 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.090099 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.092034 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.093140 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.093652 19368 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 84 similar messages]
W20250111 09:02:26.093698 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.112291 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.112435 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.112529 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.118702 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.121111 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.121412 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.124449 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.140666 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.142421 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.143801 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.146070 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.150884 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.150923 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.153688 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.171804 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.174934 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.175047 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.175803 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.179770 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.183339 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.187090 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.201365 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.206983 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.208086 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.209540 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.210875 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.213634 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.217914 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.232216 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.241199 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.242340 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.243976 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.246150 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.246284 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.252053 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.267624 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.274024 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.274065 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.276124 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.280692 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.282696 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.289314 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.302522 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.308933 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.310209 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.313627 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.316821 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.316948 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.322156 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.338145 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.343390 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.346629 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.350981 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.351477 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.351874 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.360476 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.372570 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.380302 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.382428 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.386590 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.388350 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.390350 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.397787 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.410346 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.418668 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.418710 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.425165 17457 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.425205 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.428259 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.434659 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.447813 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.456969 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.458109 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.462172 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.466279 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.468777 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.474975 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.485230 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.496404 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.496460 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.503008 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.504797 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.508471 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.518189 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.524664 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.538218 19368 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.538681 19436 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.543150 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.548002 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.550410 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.557392 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.567641 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.580310 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.581223 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.584142 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.592511 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:26.594862 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.600129 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.612345 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.623612 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:26.623919 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:02:26.624326 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:26.624426 17477 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 160 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:26.624491 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 160 FOLLOWER]: Advancing to term 161
W20250111 09:02:26.624812 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.625063 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:26.625545 17477 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 161 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:26.625860 17477 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 161 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:26.626451 17477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:26.626452 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 161 candidate_status { last_received { term: 159 index: 5504 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:26.626531 17477 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 160 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:26.626549 17337 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 160 FOLLOWER]: Advancing to term 161
I20250111 09:02:26.626571 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 160 FOLLOWER]: Advancing to term 161
I20250111 09:02:26.627125 17477 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 161 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:26.627283 17337 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 161 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 161.
I20250111 09:02:26.627398 17477 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 161 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:26.627485 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 161 candidate_status { last_received { term: 159 index: 5504 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:26.627590 17611 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 160 FOLLOWER]: Advancing to term 161
I20250111 09:02:26.627693 17612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 161 candidate_status { last_received { term: 160 index: 5570 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:26.627805 17612 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 160 FOLLOWER]: Advancing to term 161
I20250111 09:02:26.628342 17611 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 161 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 161 because replica has last-logged OpId of term: 159 index: 5507, which is greater than that of the candidate, which has last-logged OpId of term: 159 index: 5504.
I20250111 09:02:26.628428 17612 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 161 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 161.
I20250111 09:02:26.628688 17337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 161 candidate_status { last_received { term: 160 index: 5570 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:26.628686 17410 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 161 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters: b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:26.628686 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 161 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:02:26.628786 17337 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 160 LEADER]: Stepping down as leader of term 160
I20250111 09:02:26.628835 17337 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 160 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:26.628968 19454 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 161 FOLLOWER]: Leader election won for term 161
I20250111 09:02:26.629014 17337 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5570, Committed index: 5570, Last appended: 160.5570, Last appended by leader: 5570, Current term: 160, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:26.629086 19455 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 161 FOLLOWER]: Leader election won for term 161
I20250111 09:02:26.629150 19454 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 161 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:26.629204 17337 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 160 FOLLOWER]: Advancing to term 161
I20250111 09:02:26.629225 19455 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 161 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:26.629287 19454 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5570, Committed index: 5570, Last appended: 160.5570, Last appended by leader: 5570, Current term: 161, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:26.629362 19455 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5502, Committed index: 5502, Last appended: 159.5504, Last appended by leader: 5504, Current term: 161, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:26.630089 17337 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 161 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 161.
I20250111 09:02:26.631028 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 160 to 161, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 161 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:26.631227 17177 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 159 to 161, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 161 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:02:26.636072 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:26.639284 17337 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 161 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 159 index: 5494. Preceding OpId from leader: term: 161 index: 5506. (index mismatch)
I20250111 09:02:26.639344 17612 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 161 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 159 index: 5507. Preceding OpId from leader: term: 161 index: 5506. (term mismatch)
I20250111 09:02:26.639387 17612 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 5505
I20250111 09:02:26.639420 17612 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 159.5506
W20250111 09:02:26.639523 17612 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:02:26.639685 19457 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5505, Last known committed idx: 5494, Time since last communication: 0.000s
I20250111 09:02:26.639755 17612 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 159.5507
W20250111 09:02:26.639858 17612 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
I20250111 09:02:26.639955 17612 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=7114, attempt_no=0}) took 1008 ms. Trace:
I20250111 09:02:26.639994 17612 rpcz_store.cc:276] 0111 09:02:25.631295 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:25.631334 (+ 39us) service_pool.cc:225] Handling call
0111 09:02:26.639948 (+1008614us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:25.643400 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:25.643418 (+ 18us) participant_op.cc:199] PREPARE: finished
0111 09:02:25.643435 (+ 17us) participant_op.cc:212] START. Timestamp: 1736586145643390
0111 09:02:25.643437 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:02:25.643512 (+ 75us) log.cc:844] Serialized 68 byte log entry
0111 09:02:25.655911 (+ 12399us) op_driver.cc:464] REPLICATION: finished
0111 09:02:25.656028 (+ 117us) participant_op.cc:265] APPLY: starting
0111 09:02:25.656911 (+ 883us) participant_op.cc:270] APPLY: finished
0111 09:02:25.656939 (+ 28us) log.cc:844] Serialized 16 byte log entry
0111 09:02:25.656980 (+ 41us) participant_op.cc:94] Released txn lock
0111 09:02:25.656986 (+ 6us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:02:25.658896 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 2a4e17d0e2894eaa95b650f1af0ce650
0111 09:02:25.658951 (+ 55us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:25.658953 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:02:25.658954 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:25.659087 (+ 133us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:02:25.659113 (+ 26us) tablet.cc:698] Row locks acquired
0111 09:02:25.659115 (+ 2us) write_op.cc:260] PREPARE: finished
0111 09:02:25.659136 (+ 21us) write_op.cc:270] Start()
0111 09:02:25.659152 (+ 16us) write_op.cc:276] Timestamp: P: 1736586145659093 usec, L: 0
0111 09:02:25.659154 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:02:25.659285 (+ 131us) log.cc:844] Serialized 307 byte log entry
0111 09:02:26.639835 (+980550us) op_driver.cc:464] REPLICATION: finished
0111 09:02:26.639841 (+ 6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0111 09:02:26.639844 (+ 3us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:26.639851 (+ 7us) write_op.cc:454] Released schema lock
0111 09:02:26.639854 (+ 3us) write_op.cc:335] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":39,"txn-participant-registration.run_cpu_time_us":325,"txn-participant-registration.run_wall_time_us":10768,"child_traces":[["op",{"apply.queue_time_us":85,"apply.run_cpu_time_us":1016,"apply.run_wall_time_us":1147,"prepare.queue_time_us":1141,"prepare.run_cpu_time_us":152,"prepare.run_wall_time_us":521,"replication_time_us":12454,"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":1759,"prepare.run_cpu_time_us":330,"prepare.run_wall_time_us":656,"raft.queue_time_us":172,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":980648}]]}]]}
I20250111 09:02:26.640720 19457 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5505, Last known committed idx: 5504, Time since last communication: 0.000s
I20250111 09:02:26.640800 19457 consensus_queue.cc:1225] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 159.5505 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5504
I20250111 09:02:26.641319 17612 pending_rounds.cc:77] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 5504
I20250111 09:02:26.641367 17612 pending_rounds.cc:99] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 159.5505
W20250111 09:02:26.641436 17612 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
W20250111 09:02:26.646950 19449 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.648074 19453 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:26.664548 17335 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 161 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 160 index: 5570. Preceding OpId from leader: term: 161 index: 5572. (index mismatch)
I20250111 09:02:26.666136 19457 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5571, Last known committed idx: 5570, Time since last communication: 0.000s
I20250111 09:02:26.666559 17611 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 161 FOLLOWER]: Refusing update from remote peer 262623ac6df745c785ce267ea39bd265: Log matching property violated. Preceding OpId in replica: term: 160 index: 5570. Preceding OpId from leader: term: 161 index: 5572. (index mismatch)
I20250111 09:02:26.666910 19457 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5571, Last known committed idx: 5570, Time since last communication: 0.000s
W20250111 09:02:26.668534 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.710573 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:26.711696 19477 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 64 similar messages]
W20250111 09:02:26.719635 19462 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 67 similar messages]
I20250111 09:02:27.227807 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:02:27.228298 17612 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:27.228418 17612 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 161 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:27.228470 17612 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 161 FOLLOWER]: Advancing to term 162
I20250111 09:02:27.229599 17612 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 162 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:27.230000 17612 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 162 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:27.230573 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 162 candidate_status { last_received { term: 161 index: 5646 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:27.230724 17477 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 161 LEADER]: Stepping down as leader of term 161
I20250111 09:02:27.230804 17477 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 161 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:27.231060 17611 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:27.231069 17477 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5646, Committed index: 5646, Last appended: 161.5646, Last appended by leader: 5646, Current term: 161, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:27.231165 17611 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 161 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:27.231215 17611 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 161 FOLLOWER]: Advancing to term 162
I20250111 09:02:27.231209 17477 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 161 FOLLOWER]: Advancing to term 162
I20250111 09:02:27.232162 17611 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 162 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:27.232404 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 162 candidate_status { last_received { term: 161 index: 5646 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:27.232448 17611 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 162 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:27.232511 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 161 FOLLOWER]: Advancing to term 162
I20250111 09:02:27.232816 17477 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 162 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 162.
I20250111 09:02:27.233433 17336 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 162 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 162.
I20250111 09:02:27.233965 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 162 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:27.234438 19480 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 162 FOLLOWER]: Leader election won for term 162
I20250111 09:02:27.234524 19480 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 162 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:27.234634 19480 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5644, Committed index: 5644, Last appended: 161.5646, Last appended by leader: 5646, Current term: 162, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:27.234944 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 162 candidate_status { last_received { term: 161 index: 5704 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:27.235033 17477 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 161 LEADER]: Stepping down as leader of term 161
I20250111 09:02:27.235069 17477 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 161 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:27.235170 17477 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5704, Committed index: 5704, Last appended: 161.5704, Last appended by leader: 5704, Current term: 161, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:27.235268 17477 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 161 FOLLOWER]: Advancing to term 162
W20250111 09:02:27.235999 19451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:27.235987 17477 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 162 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 162.
I20250111 09:02:27.236536 17545 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 162 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
W20250111 09:02:27.236547 19450 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:27.236685 19422 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 162 FOLLOWER]: Leader election won for term 162
I20250111 09:02:27.236812 19422 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 162 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:27.236935 19422 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5704, Committed index: 5704, Last appended: 161.5704, Last appended by leader: 5704, Current term: 162, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:27.237747 19492 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:27.238471 17178 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 161 to 162, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 162 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
W20250111 09:02:27.240912 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:27.241294 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:27.244295 17454 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:27.244798 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:27.245117 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 162 candidate_status { last_received { term: 161 index: 5704 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:27.245285 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 161 FOLLOWER]: Advancing to term 162
I20250111 09:02:27.246289 17336 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 162 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 162.
W20250111 09:02:27.247129 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:27.247822 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:27.248960 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:27.254379 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:27.254717 19498 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:27.255842 19500 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:27.256253 19492 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:27.257112 17474 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 162 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 161 index: 5704. Preceding OpId from leader: term: 162 index: 5706. (index mismatch)
I20250111 09:02:27.257447 19422 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5705, Last known committed idx: 5704, Time since last communication: 0.000s
I20250111 09:02:27.257745 17476 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 162 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 161 index: 5646. Preceding OpId from leader: term: 162 index: 5648. (index mismatch)
W20250111 09:02:27.258127 19502 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:27.258319 19491 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5647, Last known committed idx: 5646, Time since last communication: 0.000s
W20250111 09:02:27.259413 19499 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:27.262367 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 161 to 162, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 162 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } } }
W20250111 09:02:27.264775 19499 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:27.267175 19492 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:27.269207 19501 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:27.271745 17338 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 162 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 161 index: 5646. Preceding OpId from leader: term: 162 index: 5648. (index mismatch)
I20250111 09:02:27.272061 17336 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 162 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 161 index: 5704. Preceding OpId from leader: term: 162 index: 5706. (index mismatch)
I20250111 09:02:27.272356 19422 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5647, Last known committed idx: 5646, Time since last communication: 0.000s
I20250111 09:02:27.284276 19422 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5705, Last known committed idx: 5704, Time since last communication: 0.000s
W20250111 09:02:27.360934 19463 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 28 similar messages]
I20250111 09:02:28.185055 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:02:28.185607 17336 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:28.185729 17336 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 162 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:28.185823 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 162 FOLLOWER]: Advancing to term 163
I20250111 09:02:28.187086 17336 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 163 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:28.187465 17336 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 163 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:28.187760 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 163 candidate_status { last_received { term: 162 index: 5863 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:28.188558 17340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:28.188700 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 163 candidate_status { last_received { term: 162 index: 5863 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:28.188750 17340 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 162 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:28.188799 17340 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 162 FOLLOWER]: Advancing to term 163
I20250111 09:02:28.188799 17613 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 162 LEADER]: Stepping down as leader of term 162
I20250111 09:02:28.188834 17613 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 162 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:28.188956 17613 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5863, Committed index: 5863, Last appended: 162.5866, Last appended by leader: 5866, Current term: 162, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:28.189111 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 162 FOLLOWER]: Advancing to term 163
I20250111 09:02:28.189721 17340 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 163 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:28.190081 17613 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 163 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 163 because replica has last-logged OpId of term: 162 index: 5866, which is greater than that of the candidate, which has last-logged OpId of term: 162 index: 5863.
I20250111 09:02:28.190075 17340 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 163 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:28.190944 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 163 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:28.191043 17477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 163 candidate_status { last_received { term: 162 index: 5947 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:28.188071 17337 raft_consensus.cc:1235] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 163 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 162. Current term is 163. Ops: [162.5864-162.5866]
I20250111 09:02:28.192394 19516 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 163 FOLLOWER]: Leader election lost for term 163. Reason: could not achieve majority
I20250111 09:02:28.192656 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 163 candidate_status { last_received { term: 162 index: 5947 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:28.192741 17613 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 162 LEADER]: Stepping down as leader of term 162
I20250111 09:02:28.192776 17613 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 162 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:28.192871 17613 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5947, Committed index: 5947, Last appended: 162.5947, Last appended by leader: 5947, Current term: 162, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:28.192932 17336 raft_consensus.cc:1235] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 163 FOLLOWER]: Rejecting Update request from peer b5fb421fa3354606aecbeafe55a0565c for earlier term 162. Current term is 163. Ops: []
I20250111 09:02:28.192953 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 162 FOLLOWER]: Advancing to term 163
I20250111 09:02:28.193691 17613 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 163 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 163.
W20250111 09:02:28.193848 19476 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.003s ago.
W20250111 09:02:28.194072 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.194109 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:28.194382 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 163 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:02:28.194622 19516 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 163 FOLLOWER]: Leader election won for term 163
I20250111 09:02:28.196802 19516 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 163 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:28.196964 19516 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5944, Committed index: 5944, Last appended: 162.5947, Last appended by leader: 5947, Current term: 163, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:28.198729 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:28.198688 17178 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 162 to 163, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 163 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:02:28.199460 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.201208 19508 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.206349 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.211309 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.214264 19521 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 7 similar messages]
W20250111 09:02:28.214318 19521 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.219239 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.224990 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.230032 19524 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 39 similar messages]
W20250111 09:02:28.230101 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.232864 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.239202 19508 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.244187 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.246014 19521 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.251892 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.256801 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.263739 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.267331 19508 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.274300 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.278021 19521 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.289043 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.291731 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.301079 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.303162 19508 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.318792 19521 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.319741 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.331475 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.335542 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.347332 19508 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.354375 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.364217 19521 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.375178 17588 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.380328 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.391944 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.398085 19508 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 72 similar messages]
W20250111 09:02:28.398151 19508 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.409787 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.416152 19521 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.431360 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.438760 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.452168 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.459445 19508 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.473111 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.482483 19521 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.497550 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.506781 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.520315 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.534583 19508 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.545049 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.561436 19521 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.573777 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.588739 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.600569 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.619176 19508 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.631408 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.648109 19521 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.663997 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.677038 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:28.687678 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 162 FOLLOWER]: Advancing to term 163
I20250111 09:02:28.688454 17467 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 163 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 162 index: 5945. Preceding OpId from leader: term: 163 index: 5948. (index mismatch)
I20250111 09:02:28.688777 19517 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5948, Last known committed idx: 5942, Time since last communication: 0.000s
I20250111 09:02:28.691556 17613 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 163 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 162 index: 5947. Preceding OpId from leader: term: 163 index: 5948. (index mismatch)
W20250111 09:02:28.695791 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:28.698659 19526 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5948, Last known committed idx: 5947, Time since last communication: 0.000s
W20250111 09:02:28.710496 19521 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.711243 19529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.711621 19530 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.714787 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.716150 19535 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.717207 19535 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.719197 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.722431 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.722792 19530 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.723803 19508 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.724131 19538 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.725724 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.727246 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.729936 19530 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.730278 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.732031 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.733659 19538 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.734998 19530 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.739151 19538 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.741624 19538 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.741734 19530 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.742986 19530 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.744393 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.748992 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.749771 19535 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.751741 19535 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.753028 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.758740 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.758826 19530 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.760574 19538 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.762527 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.766242 19530 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.768491 19530 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.768491 19538 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.773075 19530 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.775285 19535 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.778712 19535 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.778779 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.780891 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.787089 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.791649 19530 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.793923 17572 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.794204 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.798017 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.801585 19538 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.803566 19538 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.806392 19530 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.808329 19538 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.809885 19530 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.816143 19530 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.818156 19530 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.821764 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.824959 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.831120 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.834784 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.835317 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.836582 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.841207 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.845064 19530 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.846062 19538 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.850697 19538 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.854192 19530 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.856375 19530 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.861589 19530 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.866863 19530 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.873654 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.873718 19535 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.874027 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.877825 19535 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.882036 19535 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.885236 19535 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.890918 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.892468 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:28.892964 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 1
I20250111 09:02:28.893425 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:28.893543 17467 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 162 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:28.893625 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 162 FOLLOWER]: Advancing to term 163
I20250111 09:02:28.894397 17467 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 163 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:28.894729 17467 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 163 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:28.894987 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 163 candidate_status { last_received { term: 162 index: 5863 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:28.895126 17336 raft_consensus.cc:2388] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 163 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 in current term 163: Already voted for candidate fb270e42d7c9400f8c05bb9565db1972 in this term.
I20250111 09:02:28.895130 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 163 candidate_status { last_received { term: 162 index: 5863 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:28.895287 17613 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 163 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 163 because replica has last-logged OpId of term: 162 index: 5866, which is greater than that of the candidate, which has last-logged OpId of term: 162 index: 5863.
I20250111 09:02:28.895396 17467 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "262623ac6df745c785ce267ea39bd265"
from {username='slave'} at 127.0.0.1:50256
I20250111 09:02:28.895504 17467 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 163 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:28.895537 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 163 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:28.895562 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 163 FOLLOWER]: Advancing to term 164
I20250111 09:02:28.895686 19525 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 163 FOLLOWER]: Leader election lost for term 163. Reason: could not achieve majority
I20250111 09:02:28.896282 17467 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 164 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:28.896538 17467 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 164 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:28.896764 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 164 candidate_status { last_received { term: 163 index: 5955 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:28.896827 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 164 candidate_status { last_received { term: 163 index: 5955 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:28.896862 17336 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 163 LEADER]: Stepping down as leader of term 163
I20250111 09:02:28.896898 17336 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 163 LEADER]: Becoming Follower/Learner. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:28.896924 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 163 FOLLOWER]: Advancing to term 164
I20250111 09:02:28.897037 17336 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5955, Committed index: 5955, Last appended: 163.5955, Last appended by leader: 5955, Current term: 163, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:28.897171 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 163 FOLLOWER]: Advancing to term 164
I20250111 09:02:28.897668 17613 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 164 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 164.
I20250111 09:02:28.897807 17336 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 164 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262623ac6df745c785ce267ea39bd265 in term 164.
I20250111 09:02:28.897917 17408 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 164 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:02:28.898039 19525 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 164 FOLLOWER]: Leader election won for term 164
I20250111 09:02:28.898197 19525 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 164 LEADER]: Becoming Leader. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:28.898319 19525 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5955, Committed index: 5955, Last appended: 163.5955, Last appended by leader: 5955, Current term: 164, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:28.899524 17179 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 reported cstate change: term changed from 163 to 164, leader changed from fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1) to 262623ac6df745c785ce267ea39bd265 (127.14.61.2). New cstate: current_term: 164 leader_uuid: "262623ac6df745c785ce267ea39bd265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
W20250111 09:02:28.899893 19538 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.905349 19538 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.912107 19530 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.912192 19529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.912757 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.921151 19538 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.921191 19529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.926944 19529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.933725 19535 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.934661 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.943994 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.949126 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.950515 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.953706 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.957212 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.958130 19529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.968753 19538 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.973116 19538 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.975694 19529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.981946 19529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.991133 19529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.992156 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:28.996260 19529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.997764 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:28.999017 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.006309 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.016497 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.020696 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.025490 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:29.030197 17571 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:29.032122 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.039669 19538 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:29.042986 19538 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:29.045394 19538 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:29.052258 19529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.055271 19529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.067442 19529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.069617 19529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.073004 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.079129 19529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.079208 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.081936 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:29.091917 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 2
I20250111 09:02:29.092376 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:29.092491 17613 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 163 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:29.092553 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 163 FOLLOWER]: Advancing to term 164
W20250111 09:02:29.093367 19524 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:29.093489 17613 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 164 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:29.093838 17613 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 164 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:29.094094 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 164 candidate_status { last_received { term: 162 index: 5866 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:29.094173 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 164 candidate_status { last_received { term: 162 index: 5866 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:29.094209 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 163 FOLLOWER]: Advancing to term 164
I20250111 09:02:29.094274 17467 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 163 FOLLOWER]: Advancing to term 164
I20250111 09:02:29.094573 17613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
from {username='slave'} at 127.0.0.1:43276
I20250111 09:02:29.094650 17613 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 164 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:29.094691 17613 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 164 FOLLOWER]: Advancing to term 165
W20250111 09:02:29.094691 19535 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:29.095193 17336 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 164 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 164.
I20250111 09:02:29.095245 17467 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 164 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 164.
I20250111 09:02:29.095477 17547 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 164 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:29.095491 17613 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 165 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:29.095723 19537 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 164 FOLLOWER]: Leader election won for term 164
I20250111 09:02:29.095813 19537 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 164 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:29.095827 17613 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 165 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:29.095971 19537 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5863, Committed index: 5863, Last appended: 162.5866, Last appended by leader: 5866, Current term: 164, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:29.096118 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 165 candidate_status { last_received { term: 163 index: 5955 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:29.096215 17467 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 164 LEADER]: Stepping down as leader of term 164
I20250111 09:02:29.096249 17467 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 164 LEADER]: Becoming Follower/Learner. State: Replica: 262623ac6df745c785ce267ea39bd265, State: Running, Role: LEADER
I20250111 09:02:29.096374 17467 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5955, Committed index: 5955, Last appended: 164.5956, Last appended by leader: 5956, Current term: 164, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:29.096484 17467 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 164 FOLLOWER]: Advancing to term 165
I20250111 09:02:29.096685 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 165 candidate_status { last_received { term: 163 index: 5955 } } ignore_live_leader: true dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:29.096768 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 164 FOLLOWER]: Advancing to term 165
I20250111 09:02:29.097287 17467 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 165 FOLLOWER]: Leader election vote request: Denying vote to candidate b5fb421fa3354606aecbeafe55a0565c for term 165 because replica has last-logged OpId of term: 164 index: 5956, which is greater than that of the candidate, which has last-logged OpId of term: 163 index: 5955.
I20250111 09:02:29.097461 17336 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 165 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 165.
I20250111 09:02:29.097414 17178 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 162 to 164. New cstate: current_term: 164 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:29.097685 17547 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 165 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265
I20250111 09:02:29.097862 19537 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 165 FOLLOWER]: Leader election won for term 165
I20250111 09:02:29.097937 19537 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 165 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:29.098037 19537 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5955, Committed index: 5955, Last appended: 163.5955, Last appended by leader: 5955, Current term: 165, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:29.100028 17178 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 164 to 165, leader changed from 262623ac6df745c785ce267ea39bd265 (127.14.61.2) to b5fb421fa3354606aecbeafe55a0565c (127.14.61.3). New cstate: current_term: 165 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:29.108930 17336 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 164 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 162 index: 5863. Preceding OpId from leader: term: 164 index: 5868. (index mismatch)
I20250111 09:02:29.108952 17467 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 164 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 162 index: 5863. Preceding OpId from leader: term: 164 index: 5868. (index mismatch)
I20250111 09:02:29.109325 19489 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5867, Last known committed idx: 5862, Time since last communication: 0.000s
I20250111 09:02:29.109452 19537 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5867, Last known committed idx: 5860, Time since last communication: 0.000s
I20250111 09:02:29.118774 19477 mvcc.cc:204] Tried to move back new op lower bound from 7113056866747355136 to 7113056866698555392. Current Snapshot: MvccSnapshot[applied={T|T < 7113056866747355136}]
I20250111 09:02:29.122859 17335 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 165 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 163 index: 5955. Preceding OpId from leader: term: 165 index: 5957. (index mismatch)
W20250111 09:02:29.123842 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:29.123819 19537 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5956, Last known committed idx: 5955, Time since last communication: 0.000s
I20250111 09:02:29.125485 17477 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 165 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 164 index: 5956. Preceding OpId from leader: term: 165 index: 5957. (index mismatch)
W20250111 09:02:29.125931 19535 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:29.126324 19537 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5956, Last known committed idx: 5955, Time since last communication: 0.000s
I20250111 09:02:29.126420 19537 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Peer 262623ac6df745c785ce267ea39bd265 log is divergent from this leader: its last log entry 164.5956 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5955
I20250111 09:02:29.126977 19463 mvcc.cc:204] Tried to move back new op lower bound from 7113056866803359744 to 7113056866706530304. Current Snapshot: MvccSnapshot[applied={T|T < 7113056866803359744}]
I20250111 09:02:29.127461 17474 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting all ops after (but not including) 5955
I20250111 09:02:29.127506 17474 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265: Aborting uncommitted NO_OP operation due to leader change: 164.5956
I20250111 09:02:29.127534 17474 raft_consensus.cc:2883] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 165 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20250111 09:02:29.128944 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.260885 19462 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 46 similar messages]
W20250111 09:02:29.435950 19549 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 45 similar messages]
I20250111 09:02:29.931509 14580 txn_write_ops-itest.cc:638] attempting to promote replicas at tserver 0
I20250111 09:02:29.932055 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:29.932451 17338 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 164 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:29.932590 17338 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 164 FOLLOWER]: Advancing to term 165
I20250111 09:02:29.934099 17338 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 165 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:29.935407 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 165 candidate_status { last_received { term: 164 index: 6082 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:29.935350 17338 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 165 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:29.936177 17336 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd"
dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
from {username='slave'} at 127.0.0.1:39798
I20250111 09:02:29.936295 17336 raft_consensus.cc:491] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 165 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 09:02:29.936347 17336 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 165 FOLLOWER]: Advancing to term 166
I20250111 09:02:29.937180 17336 raft_consensus.cc:513] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 166 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:29.937458 17336 leader_election.cc:290] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 166 election: Requested vote from peers 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:29.937808 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 166 candidate_status { last_received { term: 165 index: 6152 } } ignore_live_leader: true dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:29.938160 17474 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 165 FOLLOWER]: Advancing to term 166
I20250111 09:02:29.938596 17614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab63c6543ad4a0ab83e76267a725edd" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 166 candidate_status { last_received { term: 165 index: 6152 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:29.938697 17614 raft_consensus.cc:3049] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 165 LEADER]: Stepping down as leader of term 165
I20250111 09:02:29.938738 17614 raft_consensus.cc:738] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 165 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:29.938763 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "fb270e42d7c9400f8c05bb9565db1972" candidate_term: 165 candidate_status { last_received { term: 164 index: 6082 } } ignore_live_leader: true dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:29.938859 17613 raft_consensus.cc:3049] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 164 LEADER]: Stepping down as leader of term 164
I20250111 09:02:29.938839 17614 consensus_queue.cc:260] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6152, Committed index: 6152, Last appended: 165.6153, Last appended by leader: 6153, Current term: 165, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:29.938896 17613 raft_consensus.cc:738] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 164 LEADER]: Becoming Follower/Learner. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:29.938941 17614 raft_consensus.cc:3054] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 165 FOLLOWER]: Advancing to term 166
I20250111 09:02:29.939023 17613 consensus_queue.cc:260] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6082, Committed index: 6082, Last appended: 164.6084, Last appended by leader: 6084, Current term: 164, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:29.939111 17613 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 164 FOLLOWER]: Advancing to term 165
I20250111 09:02:29.939810 17474 raft_consensus.cc:2463] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 166 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb270e42d7c9400f8c05bb9565db1972 in term 166.
I20250111 09:02:29.939860 17614 raft_consensus.cc:2405] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 166 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 166 because replica has last-logged OpId of term: 165 index: 6153, which is greater than that of the candidate, which has last-logged OpId of term: 165 index: 6152.
I20250111 09:02:29.940028 17613 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 165 FOLLOWER]: Leader election vote request: Denying vote to candidate fb270e42d7c9400f8c05bb9565db1972 for term 165 because replica has last-logged OpId of term: 164 index: 6084, which is greater than that of the candidate, which has last-logged OpId of term: 164 index: 6082.
I20250111 09:02:29.940114 17271 leader_election.cc:304] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 166 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, fb270e42d7c9400f8c05bb9565db1972; no voters:
I20250111 09:02:29.941886 17271 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [CANDIDATE]: Term 165 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb270e42d7c9400f8c05bb9565db1972; no voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c
I20250111 09:02:29.942014 19557 raft_consensus.cc:2798] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 166 FOLLOWER]: Leader election won for term 166
I20250111 09:02:29.942305 19557 raft_consensus.cc:695] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [term 166 LEADER]: Becoming Leader. State: Replica: fb270e42d7c9400f8c05bb9565db1972, State: Running, Role: LEADER
I20250111 09:02:29.943234 19558 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 165 FOLLOWER]: Leader election lost for term 165. Reason: could not achieve majority
I20250111 09:02:29.943650 19557 consensus_queue.cc:237] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6152, Committed index: 6152, Last appended: 165.6152, Last appended by leader: 6152, Current term: 166, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
W20250111 09:02:29.943909 17592 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:29.944022 17587 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:29.946049 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:29.946312 17177 catalog_manager.cc:5526] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 reported cstate change: term changed from 165 to 166, leader changed from b5fb421fa3354606aecbeafe55a0565c (127.14.61.3) to fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1). New cstate: current_term: 166 leader_uuid: "fb270e42d7c9400f8c05bb9565db1972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:29.947171 17476 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P 262623ac6df745c785ce267ea39bd265 [term 166 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 165 index: 6152. Preceding OpId from leader: term: 166 index: 6154. (index mismatch)
I20250111 09:02:29.947407 17613 raft_consensus.cc:1270] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c [term 166 FOLLOWER]: Refusing update from remote peer fb270e42d7c9400f8c05bb9565db1972: Log matching property violated. Preceding OpId in replica: term: 165 index: 6153. Preceding OpId from leader: term: 166 index: 6154. (index mismatch)
I20250111 09:02:29.947723 19557 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6153, Last known committed idx: 6152, Time since last communication: 0.000s
W20250111 09:02:29.947849 19555 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:29.948243 19557 consensus_queue.cc:1035] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6153, Last known committed idx: 6152, Time since last communication: 0.000s
I20250111 09:02:29.948346 19557 consensus_queue.cc:1225] T 6ab63c6543ad4a0ab83e76267a725edd P fb270e42d7c9400f8c05bb9565db1972 [LEADER]: Peer b5fb421fa3354606aecbeafe55a0565c log is divergent from this leader: its last log entry 165.6153 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6152
W20250111 09:02:29.948604 19556 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:29.949093 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:29.949496 19538 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
I20250111 09:02:29.949836 17613 pending_rounds.cc:77] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting all ops after (but not including) 6152
I20250111 09:02:29.949887 17613 pending_rounds.cc:99] T 6ab63c6543ad4a0ab83e76267a725edd P b5fb421fa3354606aecbeafe55a0565c: Aborting uncommitted WRITE_OP operation due to leader change: 165.6153
W20250111 09:02:29.949978 17613 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Aborted: Op aborted by new leader
W20250111 09:02:29.954207 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.956384 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.957710 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.960310 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:29.967195 19569 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 44 similar messages]
W20250111 09:02:29.967296 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.967221 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.969115 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.973578 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.978936 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.979694 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.980194 19538 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:29.980264 19556 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:29.981243 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:29.983672 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:29.984232 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.986474 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.988723 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.988826 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.990087 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:29.990160 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:29.990968 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.994943 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:29.995812 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:29.996721 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:29.998716 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.000720 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.001264 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.002070 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.004485 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.004832 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.012120 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.012652 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.013289 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.013634 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.013734 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.014415 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.020504 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.023156 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.024659 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.024925 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.025439 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.027938 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.032281 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.033555 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.036666 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.037928 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.038033 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.040885 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.044507 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.045703 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.049305 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.052425 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.052770 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.054260 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.054754 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.057165 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.064412 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.067991 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.068120 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.068203 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.068964 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.072860 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.080024 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.081487 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.083022 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.083993 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.085623 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.086692 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.096565 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.097510 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.097571 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.098531 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.105115 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.105512 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.113494 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.114663 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.116096 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.116190 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.123468 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.123533 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.131686 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.132721 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.133842 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.134188 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.141482 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.141489 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.150784 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.151248 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.151799 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.153879 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.160372 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.160586 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.168270 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.170783 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.170964 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.172648 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.182304 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.183166 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.189894 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.191190 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.191416 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.194816 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.202597 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.207141 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.209767 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.212236 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.212575 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.216831 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.225766 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.228142 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.233337 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.235849 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.236312 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.237980 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.248521 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.254647 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.255769 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.260841 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.261579 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.261649 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.271749 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.279747 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.280118 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.286893 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.287406 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.288194 19566 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 29 similar messages]
W20250111 09:02:30.288244 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.296329 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.302862 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.305754 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.312645 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.314098 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.314175 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.321498 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.328608 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.334565 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.339017 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.340857 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.341779 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.350880 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.358222 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.361641 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.367352 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.367759 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.368665 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.380203 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.384827 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.392802 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.394452 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.397755 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.399188 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.408649 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.413740 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.422847 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.423362 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.427433 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.430788 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.436803 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.442886 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.452049 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.454228 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.458680 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.464416 19547 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 60 similar messages]
W20250111 09:02:30.464465 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.469830 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.473538 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.484419 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.488922 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.488976 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.499051 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.503358 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.504263 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.514493 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.520437 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.521677 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.532209 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.536283 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.538457 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.547881 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.555538 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.557898 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.568760 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.570705 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.571377 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.581033 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.591562 17318 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.591642 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.605901 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.608011 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.608027 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.618383 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.627079 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.628279 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.640887 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.642900 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.642925 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.653879 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.664914 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.666787 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.678354 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.681790 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.681960 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.691293 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.702677 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.703565 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.715019 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.718044 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.720329 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.731702 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.741727 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.744333 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.752641 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.757543 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.760776 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.770117 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.784253 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.786079 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.795965 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.798647 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.803280 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.809798 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.825237 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.825884 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.836086 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.841153 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.846246 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.850317 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.865238 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.869910 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.879848 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.884898 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.888082 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.892158 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.906536 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.915508 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.925387 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.929643 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.933431 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.937390 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:30.950870 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.961869 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.971611 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.971693 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.976634 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.982776 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:30.997031 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.007791 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.015352 19568 tablet_replica.cc:1307] Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 51 similar messages]
W20250111 09:02:31.015408 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.015450 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.021281 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.026691 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.045842 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.052791 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.061800 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.062916 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.068287 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.071784 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.094208 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.097970 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.108914 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.108911 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.117254 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.118183 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.139266 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.148933 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.156368 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.157305 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.167666 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.168668 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.188685 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.196594 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.206729 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.207862 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.216737 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.217604 19570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.236079 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.247172 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.259907 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.259968 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.266779 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.268923 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.285329 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.301059 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.312108 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.312146 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.320241 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.320237 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.338611 17586 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.351550 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.363286 19570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.364473 17573 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.373034 19570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.373063 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.391496 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.405416 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.416922 19565 tablet_replica.cc:1307] Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 36 similar messages]
W20250111 09:02:31.416973 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.420292 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.424134 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.427476 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
I20250111 09:02:31.436903 19571 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 164 FOLLOWER]: Starting leader election (detected failure of leader b5fb421fa3354606aecbeafe55a0565c)
I20250111 09:02:31.437012 19571 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 164 FOLLOWER]: Advancing to term 165
I20250111 09:02:31.438143 19571 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 165 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:31.438481 19571 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 165 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), b5fb421fa3354606aecbeafe55a0565c (127.14.61.3:43257)
I20250111 09:02:31.438720 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 165 candidate_status { last_received { term: 164 index: 6082 } } ignore_live_leader: false dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:31.438877 17336 raft_consensus.cc:2388] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 165 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 in current term 165: Already voted for candidate fb270e42d7c9400f8c05bb9565db1972 in this term.
I20250111 09:02:31.438884 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "262623ac6df745c785ce267ea39bd265" candidate_term: 165 candidate_status { last_received { term: 164 index: 6082 } } ignore_live_leader: false dest_uuid: "b5fb421fa3354606aecbeafe55a0565c"
I20250111 09:02:31.439071 17613 raft_consensus.cc:2405] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 165 FOLLOWER]: Leader election vote request: Denying vote to candidate 262623ac6df745c785ce267ea39bd265 for term 165 because replica has last-logged OpId of term: 164 index: 6084, which is greater than that of the candidate, which has last-logged OpId of term: 164 index: 6082.
I20250111 09:02:31.439409 17408 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [CANDIDATE]: Term 165 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262623ac6df745c785ce267ea39bd265; no voters: b5fb421fa3354606aecbeafe55a0565c, fb270e42d7c9400f8c05bb9565db1972
I20250111 09:02:31.439586 19571 raft_consensus.cc:2743] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 165 FOLLOWER]: Leader election lost for term 165. Reason: could not achieve majority
W20250111 09:02:31.445559 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.460467 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.472755 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.473695 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.478709 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.480084 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.502015 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.516995 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.527036 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.530649 19570 tablet_replica.cc:1307] Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER: cancelling pending write operations [suppressed 23 similar messages]
W20250111 09:02:31.530715 19570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.535380 19570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.536312 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.555874 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.574752 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.583487 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.584465 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.591739 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.592864 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.610955 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.631419 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.641848 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.642856 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.648212 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.649408 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.671093 17594 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:36423: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.689904 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.701491 17570 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.702394 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.706018 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.708184 19547 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43292: Illegal state: replica b5fb421fa3354606aecbeafe55a0565c is not leader of this config: current role FOLLOWER
W20250111 09:02:31.728523 17319 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:38369: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.751487 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.761412 19566 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.762290 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.763434 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.769798 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.788847 17456 tablet_service.cc:695] failed op from {username='slave'} at 127.14.61.3:54455: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
I20250111 09:02:31.802594 19544 raft_consensus.cc:491] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 165 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:02:31.802666 19544 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 165 FOLLOWER]: Advancing to term 166
I20250111 09:02:31.803443 19544 raft_consensus.cc:513] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 166 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:31.803794 19544 leader_election.cc:290] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 166 election: Requested vote from peers fb270e42d7c9400f8c05bb9565db1972 (127.14.61.1:43619), 262623ac6df745c785ce267ea39bd265 (127.14.61.2:38607)
I20250111 09:02:31.804028 17336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 166 candidate_status { last_received { term: 164 index: 6084 } } ignore_live_leader: false dest_uuid: "fb270e42d7c9400f8c05bb9565db1972"
I20250111 09:02:31.804030 17474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4e17d0e2894eaa95b650f1af0ce650" candidate_uuid: "b5fb421fa3354606aecbeafe55a0565c" candidate_term: 166 candidate_status { last_received { term: 164 index: 6084 } } ignore_live_leader: false dest_uuid: "262623ac6df745c785ce267ea39bd265"
I20250111 09:02:31.804118 17474 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 165 FOLLOWER]: Advancing to term 166
I20250111 09:02:31.804128 17336 raft_consensus.cc:3054] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 165 FOLLOWER]: Advancing to term 166
I20250111 09:02:31.804765 17474 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 166 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 166.
I20250111 09:02:31.804813 17336 raft_consensus.cc:2463] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 166 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5fb421fa3354606aecbeafe55a0565c in term 166.
I20250111 09:02:31.805027 17545 leader_election.cc:304] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [CANDIDATE]: Term 166 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262623ac6df745c785ce267ea39bd265, b5fb421fa3354606aecbeafe55a0565c; no voters:
I20250111 09:02:31.805191 19544 raft_consensus.cc:2798] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 166 FOLLOWER]: Leader election won for term 166
I20250111 09:02:31.805362 19544 raft_consensus.cc:695] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [term 166 LEADER]: Becoming Leader. State: Replica: b5fb421fa3354606aecbeafe55a0565c, State: Running, Role: LEADER
I20250111 09:02:31.805505 19544 consensus_queue.cc:237] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6082, Committed index: 6082, Last appended: 164.6084, Last appended by leader: 6084, Current term: 166, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } }
I20250111 09:02:31.806694 17178 catalog_manager.cc:5526] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c reported cstate change: term changed from 164 to 166. New cstate: current_term: 166 leader_uuid: "b5fb421fa3354606aecbeafe55a0565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fb421fa3354606aecbeafe55a0565c" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 43257 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:31.813165 17336 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P fb270e42d7c9400f8c05bb9565db1972 [term 166 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 164 index: 6082. Preceding OpId from leader: term: 166 index: 6086. (index mismatch)
I20250111 09:02:31.813164 17474 raft_consensus.cc:1270] T 2a4e17d0e2894eaa95b650f1af0ce650 P 262623ac6df745c785ce267ea39bd265 [term 166 FOLLOWER]: Refusing update from remote peer b5fb421fa3354606aecbeafe55a0565c: Log matching property violated. Preceding OpId in replica: term: 164 index: 6082. Preceding OpId from leader: term: 166 index: 6086. (index mismatch)
I20250111 09:02:31.813511 19572 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "262623ac6df745c785ce267ea39bd265" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6085, Last known committed idx: 6082, Time since last communication: 0.000s
I20250111 09:02:31.813637 19544 consensus_queue.cc:1035] T 2a4e17d0e2894eaa95b650f1af0ce650 P b5fb421fa3354606aecbeafe55a0565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb270e42d7c9400f8c05bb9565db1972" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6085, Last known committed idx: 6082, Time since last communication: 0.000s
I20250111 09:02:31.816732 19583 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=7862, attempt_no=0}) took 1928 ms. Trace:
I20250111 09:02:31.816793 19583 rpcz_store.cc:276] 0111 09:02:29.888384 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:29.891045 (+ 2661us) service_pool.cc:225] Handling call
0111 09:02:31.816722 (+1925677us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:29.908657 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:29.908671 (+ 14us) participant_op.cc:199] PREPARE: finished
0111 09:02:29.908684 (+ 13us) participant_op.cc:212] START. Timestamp: 1736586149908678
0111 09:02:29.908685 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:02:29.908738 (+ 53us) log.cc:844] Serialized 68 byte log entry
0111 09:02:29.919875 (+ 11137us) op_driver.cc:464] REPLICATION: finished
0111 09:02:29.923161 (+ 3286us) participant_op.cc:265] APPLY: starting
0111 09:02:29.923906 (+ 745us) participant_op.cc:270] APPLY: finished
0111 09:02:29.923927 (+ 21us) log.cc:844] Serialized 16 byte log entry
0111 09:02:29.931437 (+ 7510us) participant_op.cc:94] Released txn lock
0111 09:02:29.931443 (+ 6us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:02:29.931655 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 2a4e17d0e2894eaa95b650f1af0ce650
0111 09:02:29.931706 (+ 51us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:29.931708 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:02:29.931709 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:29.931749 (+ 40us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:02:29.931775 (+ 26us) tablet.cc:698] Row locks acquired
0111 09:02:29.931776 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:02:29.931799 (+ 23us) write_op.cc:270] Start()
0111 09:02:29.932055 (+ 256us) write_op.cc:276] Timestamp: P: 1736586149931792 usec, L: 0
0111 09:02:29.932057 (+ 2us) op_driver.cc:348] REPLICATION: starting
0111 09:02:29.932162 (+ 105us) log.cc:844] Serialized 307 byte log entry
0111 09:02:31.815770 (+1883608us) op_driver.cc:464] REPLICATION: finished
0111 09:02:31.816177 (+ 407us) write_op.cc:301] APPLY: starting
0111 09:02:31.816195 (+ 18us) tablet.cc:1364] starting BulkCheckPresence
0111 09:02:31.816406 (+ 211us) tablet.cc:1367] finished BulkCheckPresence
0111 09:02:31.816408 (+ 2us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:02:31.816469 (+ 61us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:02:31.816496 (+ 27us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=128
0111 09:02:31.816499 (+ 3us) write_op.cc:312] APPLY: finished
0111 09:02:31.816536 (+ 37us) log.cc:844] Serialized 27 byte log entry
0111 09:02:31.816659 (+ 123us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:31.816669 (+ 10us) write_op.cc:454] Released schema lock
0111 09:02:31.816672 (+ 3us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"txn-participant-registration.queue_time_us":289,"txn-participant-registration.run_cpu_time_us":277,"txn-participant-registration.run_wall_time_us":17089,"child_traces":[["op",{"apply.queue_time_us":3267,"apply.run_cpu_time_us":926,"apply.run_wall_time_us":10035,"prepare.queue_time_us":105,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":100,"replication_time_us":11180,"child_traces":[["op",{"apply.queue_time_us":364,"num_ops":1,"prepare.queue_time_us":103,"prepare.run_cpu_time_us":317,"prepare.run_wall_time_us":955,"replication_time_us":1883691,"thread_start_us":102,"threads_started":1}]]}]]}
I20250111 09:02:31.817682 19580 mvcc.cc:204] Tried to move back new op lower bound from 7113056877824098304 to 7113056877796487168. Current Snapshot: MvccSnapshot[applied={T|T < 7113056870120620032}]
I20250111 09:02:31.819936 19583 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43292 (ReqId={client: 1fa3d98c5cac4c098d4877bc1f2cff00, seq_no=7875, attempt_no=0}) took 1887 ms. Trace:
I20250111 09:02:31.819999 19583 rpcz_store.cc:276] 0111 09:02:29.932181 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:29.932348 (+ 167us) service_pool.cc:225] Handling call
0111 09:02:31.819926 (+1887578us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:29.932633 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 2a4e17d0e2894eaa95b650f1af0ce650
0111 09:02:29.932689 (+ 56us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:29.932691 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:02:29.932692 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:29.932727 (+ 35us) tablet.cc:682] Acquiring locks for 1 operations
0111 09:02:29.932751 (+ 24us) tablet.cc:698] Row locks acquired
0111 09:02:29.932752 (+ 1us) write_op.cc:260] PREPARE: finished
0111 09:02:29.932776 (+ 24us) write_op.cc:270] Start()
0111 09:02:29.932793 (+ 17us) write_op.cc:276] Timestamp: P: 1736586149932768 usec, L: 0
0111 09:02:29.932796 (+ 3us) op_driver.cc:348] REPLICATION: starting
0111 09:02:29.932968 (+ 172us) log.cc:844] Serialized 307 byte log entry
0111 09:02:31.815928 (+1882960us) op_driver.cc:464] REPLICATION: finished
0111 09:02:31.818120 (+ 2192us) write_op.cc:301] APPLY: starting
0111 09:02:31.818136 (+ 16us) tablet.cc:1364] starting BulkCheckPresence
0111 09:02:31.818306 (+ 170us) tablet.cc:1367] finished BulkCheckPresence
0111 09:02:31.818308 (+ 2us) tablet.cc:1369] starting ApplyRowOperation cycle
0111 09:02:31.818339 (+ 31us) tablet.cc:1380] finished ApplyRowOperation cycle
0111 09:02:31.818372 (+ 33us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=128
0111 09:02:31.818376 (+ 4us) write_op.cc:312] APPLY: finished
0111 09:02:31.818931 (+ 555us) log.cc:844] Serialized 27 byte log entry
0111 09:02:31.819860 (+ 929us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:31.819869 (+ 9us) write_op.cc:454] Released schema lock
0111 09:02:31.819873 (+ 4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2155,"num_ops":1,"prepare.queue_time_us":72,"prepare.run_cpu_time_us":286,"prepare.run_wall_time_us":531,"replication_time_us":1883117,"thread_start_us":67,"threads_started":1}]]}
W20250111 09:02:31.822558 19568 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.822686 19569 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.823982 19596 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.829063 19596 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.857692 19596 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50258: Illegal state: replica 262623ac6df745c785ce267ea39bd265 is not leader of this config: current role FOLLOWER
W20250111 09:02:31.863394 19565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39814: Illegal state: replica fb270e42d7c9400f8c05bb9565db1972 is not leader of this config: current role FOLLOWER
W20250111 09:02:33.049345 17570 compilation_manager.cc:203] RowProjector compilation request submit failed: Service unavailable: Thread pool is at capacity (1/1 tasks running, 100/100 tasks queued)
W20250111 09:02:33.064606 17283 compilation_manager.cc:203] RowProjector compilation request submit failed: Service unavailable: Thread pool is at capacity (1/1 tasks running, 100/100 tasks queued)
I20250111 09:02:33.077481 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 17256
I20250111 09:02:33.083578 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 17393
I20250111 09:02:33.089915 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 17530
I20250111 09:02:33.096438 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 17150
2025-01-11T09:02:33Z chronyd exiting
[ OK ] TxnWriteOpsITest.FrequentElections (64453 ms)
[ RUN ] TxnWriteOpsITest.WriteOpPerf
2025-01-11T09:02:33Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-01-11T09:02:33Z Disabled control of system clock
I20250111 09:02:33.129066 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.61.62:33371
--webserver_interface=127.14.61.62
--webserver_port=0
--builtin_ntp_servers=127.14.61.20:46497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.61.62:33371
--txn_manager_enabled=true
--txn_manager_lazily_initialized=false with env {}
W20250111 09:02:33.222281 19615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:02:33.222435 19615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:02:33.222469 19615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:02:33.225054 19615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250111 09:02:33.225091 19615 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20250111 09:02:33.225104 19615 flags.cc:425] Enabled experimental flag: --txn_manager_lazily_initialized=false
W20250111 09:02:33.225131 19615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:02:33.225147 19615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250111 09:02:33.225183 19615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250111 09:02:33.228533 19615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:46497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.61.62:33371
--txn_manager_enabled=true
--txn_manager_lazily_initialized=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.61.62:33371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.61.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.19615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:02:33.229341 19615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:02:33.230304 19615 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:33.236979 19623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:33.237541 19621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:33.238338 19615 server_base.cc:1034] running on GCE node
W20250111 09:02:33.238819 19620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:33.239353 19615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:02:33.240200 19615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:02:33.243451 19615 hybrid_clock.cc:648] HybridClock initialized: now 1736586153243392 us; error 67 us; skew 500 ppm
I20250111 09:02:33.245616 19615 webserver.cc:458] Webserver started at http://127.14.61.62:37241/ using document root <none> and password file <none>
I20250111 09:02:33.246239 19615 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:33.246330 19615 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:33.246539 19615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:02:33.248288 19615 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/master-0/data/instance:
uuid: "d10d0355e7304d55833fa4e39ed0ee44"
format_stamp: "Formatted at 2025-01-11 09:02:33 on dist-test-slave-nvbt"
I20250111 09:02:33.248683 19615 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/master-0/wal/instance:
uuid: "d10d0355e7304d55833fa4e39ed0ee44"
format_stamp: "Formatted at 2025-01-11 09:02:33 on dist-test-slave-nvbt"
I20250111 09:02:33.251693 19615 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20250111 09:02:33.253506 19629 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:33.254407 19615 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:33.254498 19615 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/master-0/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/master-0/wal
uuid: "d10d0355e7304d55833fa4e39ed0ee44"
format_stamp: "Formatted at 2025-01-11 09:02:33 on dist-test-slave-nvbt"
I20250111 09:02:33.254559 19615 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:33.282467 19615 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:33.282966 19615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:02:33.283092 19615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:33.290557 19615 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:33371
I20250111 09:02:33.290565 19691 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:33371 every 8 connection(s)
I20250111 09:02:33.291494 19615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
I20250111 09:02:33.293609 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 19615
I20250111 09:02:33.293699 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/master-0/wal/instance
I20250111 09:02:33.293706 19692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:33.298700 19692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44: Bootstrap starting.
I20250111 09:02:33.301102 19692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:33.301770 19692 log.cc:826] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44: Log is configured to *not* fsync() on all Append() calls
I20250111 09:02:33.303260 19692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44: No bootstrap required, opened a new log
I20250111 09:02:33.305859 19692 raft_consensus.cc:357] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d10d0355e7304d55833fa4e39ed0ee44" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 33371 } }
I20250111 09:02:33.306046 19692 raft_consensus.cc:383] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:33.306108 19692 raft_consensus.cc:738] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d10d0355e7304d55833fa4e39ed0ee44, State: Initialized, Role: FOLLOWER
I20250111 09:02:33.306547 19692 consensus_queue.cc:260] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d10d0355e7304d55833fa4e39ed0ee44" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 33371 } }
I20250111 09:02:33.306692 19692 raft_consensus.cc:397] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:02:33.306746 19692 raft_consensus.cc:491] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:02:33.306833 19692 raft_consensus.cc:3054] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:33.307420 19692 raft_consensus.cc:513] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d10d0355e7304d55833fa4e39ed0ee44" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 33371 } }
I20250111 09:02:33.307685 19692 leader_election.cc:304] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d10d0355e7304d55833fa4e39ed0ee44; no voters:
I20250111 09:02:33.307895 19692 leader_election.cc:290] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:02:33.307971 19697 raft_consensus.cc:2798] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:33.308141 19697 raft_consensus.cc:695] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44 [term 1 LEADER]: Becoming Leader. State: Replica: d10d0355e7304d55833fa4e39ed0ee44, State: Running, Role: LEADER
I20250111 09:02:33.308377 19697 consensus_queue.cc:237] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d10d0355e7304d55833fa4e39ed0ee44" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 33371 } }
I20250111 09:02:33.308775 19692 sys_catalog.cc:564] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44 [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:02:33.309669 19698 sys_catalog.cc:455] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d10d0355e7304d55833fa4e39ed0ee44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d10d0355e7304d55833fa4e39ed0ee44" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 33371 } } }
I20250111 09:02:33.309793 19698 sys_catalog.cc:458] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44 [sys.catalog]: This master's current role is: LEADER
I20250111 09:02:33.310153 19706 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:02:33.310065 19699 sys_catalog.cc:455] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d10d0355e7304d55833fa4e39ed0ee44. Latest consensus state: current_term: 1 leader_uuid: "d10d0355e7304d55833fa4e39ed0ee44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d10d0355e7304d55833fa4e39ed0ee44" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 33371 } } }
I20250111 09:02:33.310675 19699 sys_catalog.cc:458] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44 [sys.catalog]: This master's current role is: LEADER
I20250111 09:02:33.312413 19706 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:02:33.316932 19706 catalog_manager.cc:1348] Generated new cluster ID: a9ad96b415af4c09a7574974d1ca6120
I20250111 09:02:33.317032 19706 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:02:33.324360 19706 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:02:33.325300 19706 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:02:33.332140 19706 catalog_manager.cc:5899] T 00000000000000000000000000000000 P d10d0355e7304d55833fa4e39ed0ee44: Generated new TSK 0
I20250111 09:02:33.332731 19706 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:02:33.336138 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.1:0
--local_ip_for_outbound_sockets=127.14.61.1
--webserver_interface=127.14.61.1
--webserver_port=0
--tserver_master_addrs=127.14.61.62:33371
--builtin_ntp_servers=127.14.61.20:46497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tablet_max_pending_txn_write_ops=10
--enable_txn_partition_lock=false
--enable_txn_system_client_init=true with env {}
I20250111 09:02:33.341663 19646 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47836:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20250111 09:02:33.346395 19692 master.cc:440] Invalid argument: unable to initialize TxnManager: Error creating table kudu_system.kudu_transactions on the master: not enough live tablet servers to create a table with the requested replication factor 3; 0 tablet servers are alive: unable to init TxnManager, will retry in 1.000s
W20250111 09:02:33.435281 19721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:02:33.435523 19721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:02:33.435596 19721 flags.cc:425] Enabled unsafe flag: --enable_txn_partition_lock=false
W20250111 09:02:33.435662 19721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:02:33.438511 19721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:02:33.438604 19721 flags.cc:425] Enabled experimental flag: --tablet_max_pending_txn_write_ops=10
W20250111 09:02:33.438647 19721 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:02:33.438715 19721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.1
I20250111 09:02:33.442028 19721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:46497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.61.1
--webserver_port=0
--enable_txn_partition_lock=false
--tablet_max_pending_txn_write_ops=10
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:33371
--never_fsync=true
--heap_profile_path=/tmp/kudu.19721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.1
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:02:33.442832 19721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:02:33.443869 19721 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:33.452229 19726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:33.452877 19729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:33.453491 19727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:33.454190 19721 server_base.cc:1034] running on GCE node
I20250111 09:02:33.454506 19721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:02:33.454998 19721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:02:33.456142 19721 hybrid_clock.cc:648] HybridClock initialized: now 1736586153456124 us; error 35 us; skew 500 ppm
I20250111 09:02:33.457791 19721 webserver.cc:458] Webserver started at http://127.14.61.1:33501/ using document root <none> and password file <none>
I20250111 09:02:33.458243 19721 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:33.458303 19721 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:33.458490 19721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:02:33.460027 19721 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-0/data/instance:
uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40"
format_stamp: "Formatted at 2025-01-11 09:02:33 on dist-test-slave-nvbt"
I20250111 09:02:33.460464 19721 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-0/wal/instance:
uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40"
format_stamp: "Formatted at 2025-01-11 09:02:33 on dist-test-slave-nvbt"
I20250111 09:02:33.463483 19721 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20250111 09:02:33.465286 19735 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:33.466161 19721 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:33.466269 19721 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-0/wal
uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40"
format_stamp: "Formatted at 2025-01-11 09:02:33 on dist-test-slave-nvbt"
I20250111 09:02:33.466342 19721 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:33.480875 19721 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:33.481333 19721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:02:33.481456 19721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:33.482558 19721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:02:33.482602 19721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:33.482650 19721 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:02:33.482681 19721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:33.491928 19721 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:39669
I20250111 09:02:33.492048 19855 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:39669 every 8 connection(s)
I20250111 09:02:33.492687 19721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
I20250111 09:02:33.493736 19856 heartbeater.cc:346] Connected to a master server at 127.14.61.62:33371
I20250111 09:02:33.493857 19856 heartbeater.cc:463] Registering TS with master...
I20250111 09:02:33.494277 19856 heartbeater.cc:510] Master 127.14.61.62:33371 requested a full tablet report, sending...
I20250111 09:02:33.495484 19646 ts_manager.cc:194] Registered new tserver with Master: c5ce5cefbeac43f5a1fe36c6dedfdf40 (127.14.61.1:39669)
I20250111 09:02:33.496642 19646 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.1:35473
I20250111 09:02:33.501689 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 19721
I20250111 09:02:33.501776 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-0/wal/instance
I20250111 09:02:33.503422 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.2:0
--local_ip_for_outbound_sockets=127.14.61.2
--webserver_interface=127.14.61.2
--webserver_port=0
--tserver_master_addrs=127.14.61.62:33371
--builtin_ntp_servers=127.14.61.20:46497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tablet_max_pending_txn_write_ops=10
--enable_txn_partition_lock=false
--enable_txn_system_client_init=true with env {}
W20250111 09:02:33.595573 19858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:02:33.595738 19858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:02:33.595793 19858 flags.cc:425] Enabled unsafe flag: --enable_txn_partition_lock=false
W20250111 09:02:33.595847 19858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:02:33.598412 19858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:02:33.598485 19858 flags.cc:425] Enabled experimental flag: --tablet_max_pending_txn_write_ops=10
W20250111 09:02:33.598526 19858 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:02:33.598582 19858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.2
I20250111 09:02:33.601948 19858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:46497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.61.2
--webserver_port=0
--enable_txn_partition_lock=false
--tablet_max_pending_txn_write_ops=10
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:33371
--never_fsync=true
--heap_profile_path=/tmp/kudu.19858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.2
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:02:33.602768 19858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:02:33.603636 19858 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:33.610706 19863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:33.611461 19866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:33.611864 19864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:33.612658 19858 server_base.cc:1034] running on GCE node
I20250111 09:02:33.612915 19858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:02:33.613353 19858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:02:33.614483 19858 hybrid_clock.cc:648] HybridClock initialized: now 1736586153614462 us; error 32 us; skew 500 ppm
I20250111 09:02:33.615898 19858 webserver.cc:458] Webserver started at http://127.14.61.2:32901/ using document root <none> and password file <none>
I20250111 09:02:33.616319 19858 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:33.616365 19858 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:33.616513 19858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:02:33.619490 19858 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-1/data/instance:
uuid: "c0ba5bb8d237437184fc6a02ce96fa0b"
format_stamp: "Formatted at 2025-01-11 09:02:33 on dist-test-slave-nvbt"
I20250111 09:02:33.619925 19858 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-1/wal/instance:
uuid: "c0ba5bb8d237437184fc6a02ce96fa0b"
format_stamp: "Formatted at 2025-01-11 09:02:33 on dist-test-slave-nvbt"
I20250111 09:02:33.622611 19858 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20250111 09:02:33.624274 19872 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:33.625046 19858 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20250111 09:02:33.625136 19858 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-1/wal
uuid: "c0ba5bb8d237437184fc6a02ce96fa0b"
format_stamp: "Formatted at 2025-01-11 09:02:33 on dist-test-slave-nvbt"
I20250111 09:02:33.625224 19858 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:33.643303 19858 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:33.643693 19858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:02:33.643816 19858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:33.644971 19858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:02:33.645040 19858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:33.645099 19858 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:02:33.645139 19858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:33.654393 19858 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.2:38149
I20250111 09:02:33.654454 19992 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.2:38149 every 8 connection(s)
I20250111 09:02:33.655126 19858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
I20250111 09:02:33.656589 19993 heartbeater.cc:346] Connected to a master server at 127.14.61.62:33371
I20250111 09:02:33.656798 19993 heartbeater.cc:463] Registering TS with master...
I20250111 09:02:33.657166 19993 heartbeater.cc:510] Master 127.14.61.62:33371 requested a full tablet report, sending...
I20250111 09:02:33.658290 19646 ts_manager.cc:194] Registered new tserver with Master: c0ba5bb8d237437184fc6a02ce96fa0b (127.14.61.2:38149)
I20250111 09:02:33.658737 19646 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.2:38041
I20250111 09:02:33.658993 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 19858
I20250111 09:02:33.659097 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-1/wal/instance
I20250111 09:02:33.660842 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.3:0
--local_ip_for_outbound_sockets=127.14.61.3
--webserver_interface=127.14.61.3
--webserver_port=0
--tserver_master_addrs=127.14.61.62:33371
--builtin_ntp_servers=127.14.61.20:46497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tablet_max_pending_txn_write_ops=10
--enable_txn_partition_lock=false
--enable_txn_system_client_init=true with env {}
W20250111 09:02:33.753315 19995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:02:33.753486 19995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:02:33.753538 19995 flags.cc:425] Enabled unsafe flag: --enable_txn_partition_lock=false
W20250111 09:02:33.753594 19995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:02:33.756284 19995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:02:33.756366 19995 flags.cc:425] Enabled experimental flag: --tablet_max_pending_txn_write_ops=10
W20250111 09:02:33.756399 19995 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:02:33.756462 19995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.3
I20250111 09:02:33.759511 19995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:46497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.61.3
--webserver_port=0
--enable_txn_partition_lock=false
--tablet_max_pending_txn_write_ops=10
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:33371
--never_fsync=true
--heap_profile_path=/tmp/kudu.19995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.3
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:02:33.760372 19995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:02:33.761307 19995 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:33.768501 20000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:33.769203 20001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:33.769937 20003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:33.770601 19995 server_base.cc:1034] running on GCE node
I20250111 09:02:33.770884 19995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:02:33.771313 19995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:02:33.772444 19995 hybrid_clock.cc:648] HybridClock initialized: now 1736586153772426 us; error 28 us; skew 500 ppm
I20250111 09:02:33.773993 19995 webserver.cc:458] Webserver started at http://127.14.61.3:37689/ using document root <none> and password file <none>
I20250111 09:02:33.774456 19995 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:33.774502 19995 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:33.774668 19995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:02:33.776057 19995 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-2/data/instance:
uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d"
format_stamp: "Formatted at 2025-01-11 09:02:33 on dist-test-slave-nvbt"
I20250111 09:02:33.776465 19995 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-2/wal/instance:
uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d"
format_stamp: "Formatted at 2025-01-11 09:02:33 on dist-test-slave-nvbt"
I20250111 09:02:33.779291 19995 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20250111 09:02:33.781147 20009 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:33.781940 19995 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:33.782027 19995 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-2/wal
uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d"
format_stamp: "Formatted at 2025-01-11 09:02:33 on dist-test-slave-nvbt"
I20250111 09:02:33.782094 19995 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:33.795738 19995 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:33.796140 19995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:02:33.796283 19995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:33.797482 19995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:02:33.797544 19995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:33.797596 19995 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:02:33.797629 19995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:33.807516 19995 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.3:36221
I20250111 09:02:33.807554 20129 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.3:36221 every 8 connection(s)
I20250111 09:02:33.808223 19995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
I20250111 09:02:33.809206 20130 heartbeater.cc:346] Connected to a master server at 127.14.61.62:33371
I20250111 09:02:33.809334 20130 heartbeater.cc:463] Registering TS with master...
I20250111 09:02:33.809705 20130 heartbeater.cc:510] Master 127.14.61.62:33371 requested a full tablet report, sending...
I20250111 09:02:33.810475 19646 ts_manager.cc:194] Registered new tserver with Master: 0cfd85d83bd14a6a9388eb4814fbbd9d (127.14.61.3:36221)
I20250111 09:02:33.811007 19646 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.3:55557
I20250111 09:02:33.815275 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 19995
I20250111 09:02:33.815361 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpPerf.1736586070622223-14580-0/minicluster-data/ts-2/wal/instance
I20250111 09:02:33.817101 14580 external_mini_cluster.cc:934] 3 TS(s) registered with all masters
I20250111 09:02:33.824833 19646 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47850:
name: "txn_write_ops_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
W20250111 09:02:33.825449 19646 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table txn_write_ops_test in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250111 09:02:33.839558 19785 tablet_service.cc:1467] Processing CreateTablet for tablet f9c6d25609134cb58c4a74c15daa6824 (DEFAULT_TABLE table=txn_write_ops_test [id=f4ff02051c794072a48bb327d3963388]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:33.839558 19922 tablet_service.cc:1467] Processing CreateTablet for tablet f9c6d25609134cb58c4a74c15daa6824 (DEFAULT_TABLE table=txn_write_ops_test [id=f4ff02051c794072a48bb327d3963388]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:33.840430 19922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9c6d25609134cb58c4a74c15daa6824. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:33.840430 19785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9c6d25609134cb58c4a74c15daa6824. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:33.839558 19917 tablet_service.cc:1467] Processing CreateTablet for tablet 203944a7a4194f69822687d24790e320 (DEFAULT_TABLE table=txn_write_ops_test [id=f4ff02051c794072a48bb327d3963388]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:33.840831 19917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 203944a7a4194f69822687d24790e320. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:33.842079 20058 tablet_service.cc:1467] Processing CreateTablet for tablet f9c6d25609134cb58c4a74c15daa6824 (DEFAULT_TABLE table=txn_write_ops_test [id=f4ff02051c794072a48bb327d3963388]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:33.842079 20059 tablet_service.cc:1467] Processing CreateTablet for tablet 203944a7a4194f69822687d24790e320 (DEFAULT_TABLE table=txn_write_ops_test [id=f4ff02051c794072a48bb327d3963388]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:33.842832 20058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9c6d25609134cb58c4a74c15daa6824. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:33.843578 20059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 203944a7a4194f69822687d24790e320. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:33.845328 19784 tablet_service.cc:1467] Processing CreateTablet for tablet 203944a7a4194f69822687d24790e320 (DEFAULT_TABLE table=txn_write_ops_test [id=f4ff02051c794072a48bb327d3963388]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:33.845553 19784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 203944a7a4194f69822687d24790e320. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:33.847708 20145 tablet_bootstrap.cc:492] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40: Bootstrap starting.
I20250111 09:02:33.848137 20144 tablet_bootstrap.cc:492] T f9c6d25609134cb58c4a74c15daa6824 P c0ba5bb8d237437184fc6a02ce96fa0b: Bootstrap starting.
I20250111 09:02:33.848449 20146 tablet_bootstrap.cc:492] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d: Bootstrap starting.
I20250111 09:02:33.850158 20145 tablet_bootstrap.cc:654] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:33.850423 20144 tablet_bootstrap.cc:654] T f9c6d25609134cb58c4a74c15daa6824 P c0ba5bb8d237437184fc6a02ce96fa0b: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:33.850692 20146 tablet_bootstrap.cc:654] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:33.850998 20145 log.cc:826] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40: Log is configured to *not* fsync() on all Append() calls
I20250111 09:02:33.851090 20144 log.cc:826] T f9c6d25609134cb58c4a74c15daa6824 P c0ba5bb8d237437184fc6a02ce96fa0b: Log is configured to *not* fsync() on all Append() calls
I20250111 09:02:33.851404 20146 log.cc:826] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d: Log is configured to *not* fsync() on all Append() calls
I20250111 09:02:33.852550 20145 tablet_bootstrap.cc:492] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40: No bootstrap required, opened a new log
I20250111 09:02:33.852700 20145 ts_tablet_manager.cc:1397] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20250111 09:02:33.852777 20146 tablet_bootstrap.cc:492] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d: No bootstrap required, opened a new log
I20250111 09:02:33.852914 20146 ts_tablet_manager.cc:1397] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20250111 09:02:33.853826 20144 tablet_bootstrap.cc:492] T f9c6d25609134cb58c4a74c15daa6824 P c0ba5bb8d237437184fc6a02ce96fa0b: No bootstrap required, opened a new log
I20250111 09:02:33.853996 20144 ts_tablet_manager.cc:1397] T f9c6d25609134cb58c4a74c15daa6824 P c0ba5bb8d237437184fc6a02ce96fa0b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20250111 09:02:33.855576 20146 raft_consensus.cc:357] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } }
I20250111 09:02:33.855576 20145 raft_consensus.cc:357] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } }
I20250111 09:02:33.855753 20145 raft_consensus.cc:383] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:33.855753 20146 raft_consensus.cc:383] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:33.855806 20146 raft_consensus.cc:738] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cfd85d83bd14a6a9388eb4814fbbd9d, State: Initialized, Role: FOLLOWER
I20250111 09:02:33.855806 20145 raft_consensus.cc:738] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5ce5cefbeac43f5a1fe36c6dedfdf40, State: Initialized, Role: FOLLOWER
I20250111 09:02:33.856140 20144 raft_consensus.cc:357] T f9c6d25609134cb58c4a74c15daa6824 P c0ba5bb8d237437184fc6a02ce96fa0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } }
I20250111 09:02:33.856186 20145 consensus_queue.cc:260] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } }
I20250111 09:02:33.856186 20146 consensus_queue.cc:260] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } }
I20250111 09:02:33.856269 20144 raft_consensus.cc:383] T f9c6d25609134cb58c4a74c15daa6824 P c0ba5bb8d237437184fc6a02ce96fa0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:33.856316 20144 raft_consensus.cc:738] T f9c6d25609134cb58c4a74c15daa6824 P c0ba5bb8d237437184fc6a02ce96fa0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0ba5bb8d237437184fc6a02ce96fa0b, State: Initialized, Role: FOLLOWER
I20250111 09:02:33.856712 19856 heartbeater.cc:502] Master 127.14.61.62:33371 was elected leader, sending a full tablet report...
I20250111 09:02:33.856688 20144 consensus_queue.cc:260] T f9c6d25609134cb58c4a74c15daa6824 P c0ba5bb8d237437184fc6a02ce96fa0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } }
I20250111 09:02:33.856875 20145 ts_tablet_manager.cc:1428] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20250111 09:02:33.857060 20130 heartbeater.cc:502] Master 127.14.61.62:33371 was elected leader, sending a full tablet report...
I20250111 09:02:33.857129 20145 tablet_bootstrap.cc:492] T f9c6d25609134cb58c4a74c15daa6824 P c5ce5cefbeac43f5a1fe36c6dedfdf40: Bootstrap starting.
I20250111 09:02:33.857318 19993 heartbeater.cc:502] Master 127.14.61.62:33371 was elected leader, sending a full tablet report...
I20250111 09:02:33.857391 20144 ts_tablet_manager.cc:1428] T f9c6d25609134cb58c4a74c15daa6824 P c0ba5bb8d237437184fc6a02ce96fa0b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20250111 09:02:33.857589 20144 tablet_bootstrap.cc:492] T 203944a7a4194f69822687d24790e320 P c0ba5bb8d237437184fc6a02ce96fa0b: Bootstrap starting.
I20250111 09:02:33.858345 20144 tablet_bootstrap.cc:654] T 203944a7a4194f69822687d24790e320 P c0ba5bb8d237437184fc6a02ce96fa0b: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:33.858676 20145 tablet_bootstrap.cc:654] T f9c6d25609134cb58c4a74c15daa6824 P c5ce5cefbeac43f5a1fe36c6dedfdf40: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:33.859163 20144 tablet_bootstrap.cc:492] T 203944a7a4194f69822687d24790e320 P c0ba5bb8d237437184fc6a02ce96fa0b: No bootstrap required, opened a new log
I20250111 09:02:33.859242 20144 ts_tablet_manager.cc:1397] T 203944a7a4194f69822687d24790e320 P c0ba5bb8d237437184fc6a02ce96fa0b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:02:33.859694 20144 raft_consensus.cc:357] T 203944a7a4194f69822687d24790e320 P c0ba5bb8d237437184fc6a02ce96fa0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } }
I20250111 09:02:33.859745 20145 tablet_bootstrap.cc:492] T f9c6d25609134cb58c4a74c15daa6824 P c5ce5cefbeac43f5a1fe36c6dedfdf40: No bootstrap required, opened a new log
I20250111 09:02:33.859774 20144 raft_consensus.cc:383] T 203944a7a4194f69822687d24790e320 P c0ba5bb8d237437184fc6a02ce96fa0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:33.859812 20145 ts_tablet_manager.cc:1397] T f9c6d25609134cb58c4a74c15daa6824 P c5ce5cefbeac43f5a1fe36c6dedfdf40: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250111 09:02:33.859810 20144 raft_consensus.cc:738] T 203944a7a4194f69822687d24790e320 P c0ba5bb8d237437184fc6a02ce96fa0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0ba5bb8d237437184fc6a02ce96fa0b, State: Initialized, Role: FOLLOWER
I20250111 09:02:33.859936 20144 consensus_queue.cc:260] T 203944a7a4194f69822687d24790e320 P c0ba5bb8d237437184fc6a02ce96fa0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } }
I20250111 09:02:33.860198 20144 ts_tablet_manager.cc:1428] T 203944a7a4194f69822687d24790e320 P c0ba5bb8d237437184fc6a02ce96fa0b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:02:33.860425 20145 raft_consensus.cc:357] T f9c6d25609134cb58c4a74c15daa6824 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } }
I20250111 09:02:33.860534 20145 raft_consensus.cc:383] T f9c6d25609134cb58c4a74c15daa6824 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:33.860587 20145 raft_consensus.cc:738] T f9c6d25609134cb58c4a74c15daa6824 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5ce5cefbeac43f5a1fe36c6dedfdf40, State: Initialized, Role: FOLLOWER
I20250111 09:02:33.860765 20145 consensus_queue.cc:260] T f9c6d25609134cb58c4a74c15daa6824 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } }
I20250111 09:02:33.861056 20145 ts_tablet_manager.cc:1428] T f9c6d25609134cb58c4a74c15daa6824 P c5ce5cefbeac43f5a1fe36c6dedfdf40: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20250111 09:02:33.861757 20146 ts_tablet_manager.cc:1428] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20250111 09:02:33.861948 20146 tablet_bootstrap.cc:492] T 203944a7a4194f69822687d24790e320 P 0cfd85d83bd14a6a9388eb4814fbbd9d: Bootstrap starting.
I20250111 09:02:33.862677 20146 tablet_bootstrap.cc:654] T 203944a7a4194f69822687d24790e320 P 0cfd85d83bd14a6a9388eb4814fbbd9d: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:33.863581 20146 tablet_bootstrap.cc:492] T 203944a7a4194f69822687d24790e320 P 0cfd85d83bd14a6a9388eb4814fbbd9d: No bootstrap required, opened a new log
I20250111 09:02:33.863641 20146 ts_tablet_manager.cc:1397] T 203944a7a4194f69822687d24790e320 P 0cfd85d83bd14a6a9388eb4814fbbd9d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:02:33.863945 20146 raft_consensus.cc:357] T 203944a7a4194f69822687d24790e320 P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } }
I20250111 09:02:33.864018 20146 raft_consensus.cc:383] T 203944a7a4194f69822687d24790e320 P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:33.864048 20146 raft_consensus.cc:738] T 203944a7a4194f69822687d24790e320 P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cfd85d83bd14a6a9388eb4814fbbd9d, State: Initialized, Role: FOLLOWER
I20250111 09:02:33.864166 20146 consensus_queue.cc:260] T 203944a7a4194f69822687d24790e320 P 0cfd85d83bd14a6a9388eb4814fbbd9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } }
I20250111 09:02:33.864367 20146 ts_tablet_manager.cc:1428] T 203944a7a4194f69822687d24790e320 P 0cfd85d83bd14a6a9388eb4814fbbd9d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250111 09:02:33.898416 20151 raft_consensus.cc:491] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:02:33.898547 20151 raft_consensus.cc:513] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } }
I20250111 09:02:33.899379 20151 leader_election.cc:290] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5ce5cefbeac43f5a1fe36c6dedfdf40 (127.14.61.1:39669), c0ba5bb8d237437184fc6a02ce96fa0b (127.14.61.2:38149)
I20250111 09:02:33.902638 19805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c6d25609134cb58c4a74c15daa6824" candidate_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" is_pre_election: true
I20250111 09:02:33.902649 19939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c6d25609134cb58c4a74c15daa6824" candidate_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" is_pre_election: true
I20250111 09:02:33.902824 19939 raft_consensus.cc:2463] T f9c6d25609134cb58c4a74c15daa6824 P c0ba5bb8d237437184fc6a02ce96fa0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cfd85d83bd14a6a9388eb4814fbbd9d in term 0.
I20250111 09:02:33.902819 19805 raft_consensus.cc:2463] T f9c6d25609134cb58c4a74c15daa6824 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cfd85d83bd14a6a9388eb4814fbbd9d in term 0.
I20250111 09:02:33.903167 20010 leader_election.cc:304] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cfd85d83bd14a6a9388eb4814fbbd9d, c5ce5cefbeac43f5a1fe36c6dedfdf40; no voters:
I20250111 09:02:33.903347 20151 raft_consensus.cc:2798] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:02:33.903409 20151 raft_consensus.cc:491] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:02:33.903452 20151 raft_consensus.cc:3054] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:33.904117 20151 raft_consensus.cc:513] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } }
I20250111 09:02:33.904374 20151 leader_election.cc:290] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d [CANDIDATE]: Term 1 election: Requested vote from peers c5ce5cefbeac43f5a1fe36c6dedfdf40 (127.14.61.1:39669), c0ba5bb8d237437184fc6a02ce96fa0b (127.14.61.2:38149)
I20250111 09:02:33.904696 19805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c6d25609134cb58c4a74c15daa6824" candidate_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40"
I20250111 09:02:33.904698 19939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c6d25609134cb58c4a74c15daa6824" candidate_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b"
I20250111 09:02:33.904771 19939 raft_consensus.cc:3054] T f9c6d25609134cb58c4a74c15daa6824 P c0ba5bb8d237437184fc6a02ce96fa0b [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:33.904780 19805 raft_consensus.cc:3054] T f9c6d25609134cb58c4a74c15daa6824 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:33.905547 19939 raft_consensus.cc:2463] T f9c6d25609134cb58c4a74c15daa6824 P c0ba5bb8d237437184fc6a02ce96fa0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cfd85d83bd14a6a9388eb4814fbbd9d in term 1.
I20250111 09:02:33.905676 19805 raft_consensus.cc:2463] T f9c6d25609134cb58c4a74c15daa6824 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cfd85d83bd14a6a9388eb4814fbbd9d in term 1.
I20250111 09:02:33.905820 20010 leader_election.cc:304] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cfd85d83bd14a6a9388eb4814fbbd9d, c0ba5bb8d237437184fc6a02ce96fa0b; no voters:
W20250111 09:02:33.905889 19994 tablet.cc:2367] T 203944a7a4194f69822687d24790e320 P c0ba5bb8d237437184fc6a02ce96fa0b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250111 09:02:33.905977 20151 raft_consensus.cc:2798] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:33.906142 20151 raft_consensus.cc:695] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 1 LEADER]: Becoming Leader. State: Replica: 0cfd85d83bd14a6a9388eb4814fbbd9d, State: Running, Role: LEADER
I20250111 09:02:33.906430 20151 consensus_queue.cc:237] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } }
I20250111 09:02:33.908643 19645 catalog_manager.cc:5526] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cfd85d83bd14a6a9388eb4814fbbd9d (127.14.61.3). New cstate: current_term: 1 leader_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:33.923394 20150 raft_consensus.cc:491] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:02:33.923466 20150 raft_consensus.cc:513] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } }
I20250111 09:02:33.924194 20150 leader_election.cc:290] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0ba5bb8d237437184fc6a02ce96fa0b (127.14.61.2:38149), 0cfd85d83bd14a6a9388eb4814fbbd9d (127.14.61.3:36221)
I20250111 09:02:33.927235 19939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "203944a7a4194f69822687d24790e320" candidate_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" is_pre_election: true
I20250111 09:02:33.927336 19939 raft_consensus.cc:2463] T 203944a7a4194f69822687d24790e320 P c0ba5bb8d237437184fc6a02ce96fa0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5ce5cefbeac43f5a1fe36c6dedfdf40 in term 0.
I20250111 09:02:33.927459 20079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "203944a7a4194f69822687d24790e320" candidate_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" is_pre_election: true
I20250111 09:02:33.927595 20079 raft_consensus.cc:2463] T 203944a7a4194f69822687d24790e320 P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5ce5cefbeac43f5a1fe36c6dedfdf40 in term 0.
I20250111 09:02:33.927611 19736 leader_election.cc:304] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c0ba5bb8d237437184fc6a02ce96fa0b, c5ce5cefbeac43f5a1fe36c6dedfdf40; no voters:
I20250111 09:02:33.927790 20150 raft_consensus.cc:2798] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:02:33.927843 20150 raft_consensus.cc:491] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:02:33.927883 20150 raft_consensus.cc:3054] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:33.928483 20150 raft_consensus.cc:513] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } }
I20250111 09:02:33.928766 20150 leader_election.cc:290] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [CANDIDATE]: Term 1 election: Requested vote from peers c0ba5bb8d237437184fc6a02ce96fa0b (127.14.61.2:38149), 0cfd85d83bd14a6a9388eb4814fbbd9d (127.14.61.3:36221)
I20250111 09:02:33.929061 20079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "203944a7a4194f69822687d24790e320" candidate_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d"
I20250111 09:02:33.929052 19939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "203944a7a4194f69822687d24790e320" candidate_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b"
I20250111 09:02:33.929145 20079 raft_consensus.cc:3054] T 203944a7a4194f69822687d24790e320 P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:33.929143 19939 raft_consensus.cc:3054] T 203944a7a4194f69822687d24790e320 P c0ba5bb8d237437184fc6a02ce96fa0b [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:33.929744 19939 raft_consensus.cc:2463] T 203944a7a4194f69822687d24790e320 P c0ba5bb8d237437184fc6a02ce96fa0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5ce5cefbeac43f5a1fe36c6dedfdf40 in term 1.
I20250111 09:02:33.929777 20079 raft_consensus.cc:2463] T 203944a7a4194f69822687d24790e320 P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5ce5cefbeac43f5a1fe36c6dedfdf40 in term 1.
I20250111 09:02:33.930025 19739 leader_election.cc:304] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cfd85d83bd14a6a9388eb4814fbbd9d, c5ce5cefbeac43f5a1fe36c6dedfdf40; no voters:
I20250111 09:02:33.930178 20150 raft_consensus.cc:2798] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:33.930371 20150 raft_consensus.cc:695] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 1 LEADER]: Becoming Leader. State: Replica: c5ce5cefbeac43f5a1fe36c6dedfdf40, State: Running, Role: LEADER
I20250111 09:02:33.930611 20150 consensus_queue.cc:237] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } }
I20250111 09:02:33.932818 19645 catalog_manager.cc:5526] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5ce5cefbeac43f5a1fe36c6dedfdf40 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } health_report { overall_health: UNKNOWN } } }
W20250111 09:02:33.993587 19857 tablet.cc:2367] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250111 09:02:34.004279 20164 txn_manager_proxy_rpc.cc:150] re-attempting BeginTransaction request to one of TxnManagers
W20250111 09:02:34.058943 20131 tablet.cc:2367] T 203944a7a4194f69822687d24790e320 P 0cfd85d83bd14a6a9388eb4814fbbd9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250111 09:02:34.353135 19645 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47940:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20250111 09:02:34.353598 19645 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250111 09:02:34.357468 20058 tablet_service.cc:1467] Processing CreateTablet for tablet 883afaedb39b40bd94599fb96d9889fd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3c7c2f91c2fa405695c264eee974db6d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:02:34.357414 19922 tablet_service.cc:1467] Processing CreateTablet for tablet 883afaedb39b40bd94599fb96d9889fd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3c7c2f91c2fa405695c264eee974db6d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:02:34.357414 19785 tablet_service.cc:1467] Processing CreateTablet for tablet 883afaedb39b40bd94599fb96d9889fd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3c7c2f91c2fa405695c264eee974db6d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:02:34.357968 19785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 883afaedb39b40bd94599fb96d9889fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:34.357968 19922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 883afaedb39b40bd94599fb96d9889fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:34.357969 20058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 883afaedb39b40bd94599fb96d9889fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:34.360054 20146 tablet_bootstrap.cc:492] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d: Bootstrap starting.
I20250111 09:02:34.360318 20144 tablet_bootstrap.cc:492] T 883afaedb39b40bd94599fb96d9889fd P c0ba5bb8d237437184fc6a02ce96fa0b: Bootstrap starting.
I20250111 09:02:34.360383 20145 tablet_bootstrap.cc:492] T 883afaedb39b40bd94599fb96d9889fd P c5ce5cefbeac43f5a1fe36c6dedfdf40: Bootstrap starting.
I20250111 09:02:34.361085 20146 tablet_bootstrap.cc:654] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:34.361176 20144 tablet_bootstrap.cc:654] T 883afaedb39b40bd94599fb96d9889fd P c0ba5bb8d237437184fc6a02ce96fa0b: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:34.361410 20145 tablet_bootstrap.cc:654] T 883afaedb39b40bd94599fb96d9889fd P c5ce5cefbeac43f5a1fe36c6dedfdf40: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:34.362166 20146 tablet_bootstrap.cc:492] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d: No bootstrap required, opened a new log
I20250111 09:02:34.362253 20146 ts_tablet_manager.cc:1397] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20250111 09:02:34.362416 20145 tablet_bootstrap.cc:492] T 883afaedb39b40bd94599fb96d9889fd P c5ce5cefbeac43f5a1fe36c6dedfdf40: No bootstrap required, opened a new log
I20250111 09:02:34.362504 20145 ts_tablet_manager.cc:1397] T 883afaedb39b40bd94599fb96d9889fd P c5ce5cefbeac43f5a1fe36c6dedfdf40: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:34.362738 20146 raft_consensus.cc:357] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } }
I20250111 09:02:34.362819 20146 raft_consensus.cc:383] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:34.362839 20146 raft_consensus.cc:738] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cfd85d83bd14a6a9388eb4814fbbd9d, State: Initialized, Role: FOLLOWER
I20250111 09:02:34.362919 20145 raft_consensus.cc:357] T 883afaedb39b40bd94599fb96d9889fd P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } }
I20250111 09:02:34.362962 20146 consensus_queue.cc:260] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } }
I20250111 09:02:34.363022 20145 raft_consensus.cc:383] T 883afaedb39b40bd94599fb96d9889fd P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:34.363065 20145 raft_consensus.cc:738] T 883afaedb39b40bd94599fb96d9889fd P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5ce5cefbeac43f5a1fe36c6dedfdf40, State: Initialized, Role: FOLLOWER
I20250111 09:02:34.363179 20146 ts_tablet_manager.cc:1428] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250111 09:02:34.363157 20151 tablet_replica.cc:440] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } } }
I20250111 09:02:34.363189 20145 consensus_queue.cc:260] T 883afaedb39b40bd94599fb96d9889fd P c5ce5cefbeac43f5a1fe36c6dedfdf40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } }
I20250111 09:02:34.363263 20151 tablet_replica.cc:443] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:34.363459 20145 ts_tablet_manager.cc:1428] T 883afaedb39b40bd94599fb96d9889fd P c5ce5cefbeac43f5a1fe36c6dedfdf40: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:02:34.363741 20144 tablet_bootstrap.cc:492] T 883afaedb39b40bd94599fb96d9889fd P c0ba5bb8d237437184fc6a02ce96fa0b: No bootstrap required, opened a new log
I20250111 09:02:34.363709 20150 tablet_replica.cc:440] T 883afaedb39b40bd94599fb96d9889fd P c5ce5cefbeac43f5a1fe36c6dedfdf40: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } } }
I20250111 09:02:34.363808 20150 tablet_replica.cc:443] T 883afaedb39b40bd94599fb96d9889fd P c5ce5cefbeac43f5a1fe36c6dedfdf40: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:34.363809 20144 ts_tablet_manager.cc:1397] T 883afaedb39b40bd94599fb96d9889fd P c0ba5bb8d237437184fc6a02ce96fa0b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250111 09:02:34.364174 20144 raft_consensus.cc:357] T 883afaedb39b40bd94599fb96d9889fd P c0ba5bb8d237437184fc6a02ce96fa0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } }
I20250111 09:02:34.364297 20144 raft_consensus.cc:383] T 883afaedb39b40bd94599fb96d9889fd P c0ba5bb8d237437184fc6a02ce96fa0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:34.364347 20144 raft_consensus.cc:738] T 883afaedb39b40bd94599fb96d9889fd P c0ba5bb8d237437184fc6a02ce96fa0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0ba5bb8d237437184fc6a02ce96fa0b, State: Initialized, Role: FOLLOWER
I20250111 09:02:34.364454 20144 consensus_queue.cc:260] T 883afaedb39b40bd94599fb96d9889fd P c0ba5bb8d237437184fc6a02ce96fa0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } }
I20250111 09:02:34.364769 20144 ts_tablet_manager.cc:1428] T 883afaedb39b40bd94599fb96d9889fd P c0ba5bb8d237437184fc6a02ce96fa0b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:02:34.364784 20211 tablet_replica.cc:440] T 883afaedb39b40bd94599fb96d9889fd P c0ba5bb8d237437184fc6a02ce96fa0b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } } }
I20250111 09:02:34.365108 20211 tablet_replica.cc:443] T 883afaedb39b40bd94599fb96d9889fd P c0ba5bb8d237437184fc6a02ce96fa0b: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:34.391076 20151 raft_consensus.cc:491] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:02:34.391147 20151 raft_consensus.cc:513] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } }
I20250111 09:02:34.391422 20151 leader_election.cc:290] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5ce5cefbeac43f5a1fe36c6dedfdf40 (127.14.61.1:39669), c0ba5bb8d237437184fc6a02ce96fa0b (127.14.61.2:38149)
I20250111 09:02:34.391743 19805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "883afaedb39b40bd94599fb96d9889fd" candidate_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" is_pre_election: true
I20250111 09:02:34.391785 19939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "883afaedb39b40bd94599fb96d9889fd" candidate_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" is_pre_election: true
I20250111 09:02:34.391871 19805 raft_consensus.cc:2463] T 883afaedb39b40bd94599fb96d9889fd P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cfd85d83bd14a6a9388eb4814fbbd9d in term 0.
I20250111 09:02:34.391889 19939 raft_consensus.cc:2463] T 883afaedb39b40bd94599fb96d9889fd P c0ba5bb8d237437184fc6a02ce96fa0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cfd85d83bd14a6a9388eb4814fbbd9d in term 0.
I20250111 09:02:34.392098 20010 leader_election.cc:304] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cfd85d83bd14a6a9388eb4814fbbd9d, c5ce5cefbeac43f5a1fe36c6dedfdf40; no voters:
I20250111 09:02:34.392261 20151 raft_consensus.cc:2798] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:02:34.392328 20151 raft_consensus.cc:491] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:02:34.392367 20151 raft_consensus.cc:3054] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:34.392931 20151 raft_consensus.cc:513] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } }
I20250111 09:02:34.393180 20151 leader_election.cc:290] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d [CANDIDATE]: Term 1 election: Requested vote from peers c5ce5cefbeac43f5a1fe36c6dedfdf40 (127.14.61.1:39669), c0ba5bb8d237437184fc6a02ce96fa0b (127.14.61.2:38149)
I20250111 09:02:34.393456 19805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "883afaedb39b40bd94599fb96d9889fd" candidate_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40"
I20250111 09:02:34.393505 19939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "883afaedb39b40bd94599fb96d9889fd" candidate_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b"
I20250111 09:02:34.393550 19805 raft_consensus.cc:3054] T 883afaedb39b40bd94599fb96d9889fd P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:34.393599 19939 raft_consensus.cc:3054] T 883afaedb39b40bd94599fb96d9889fd P c0ba5bb8d237437184fc6a02ce96fa0b [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:34.394173 19805 raft_consensus.cc:2463] T 883afaedb39b40bd94599fb96d9889fd P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cfd85d83bd14a6a9388eb4814fbbd9d in term 1.
I20250111 09:02:34.394181 19939 raft_consensus.cc:2463] T 883afaedb39b40bd94599fb96d9889fd P c0ba5bb8d237437184fc6a02ce96fa0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cfd85d83bd14a6a9388eb4814fbbd9d in term 1.
I20250111 09:02:34.394367 20010 leader_election.cc:304] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cfd85d83bd14a6a9388eb4814fbbd9d, c0ba5bb8d237437184fc6a02ce96fa0b; no voters:
I20250111 09:02:34.394498 20151 raft_consensus.cc:2798] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:34.394577 20151 raft_consensus.cc:695] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d [term 1 LEADER]: Becoming Leader. State: Replica: 0cfd85d83bd14a6a9388eb4814fbbd9d, State: Running, Role: LEADER
I20250111 09:02:34.394685 20151 consensus_queue.cc:237] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } }
I20250111 09:02:34.395164 20157 tablet_replica.cc:440] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d: TxnStatusTablet state changed. Reason: New leader 0cfd85d83bd14a6a9388eb4814fbbd9d. Latest consensus state: current_term: 1 leader_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } } }
I20250111 09:02:34.395249 20157 tablet_replica.cc:443] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:02:34.395445 20213 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:02:34.395627 20213 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:02:34.395906 19646 catalog_manager.cc:5526] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cfd85d83bd14a6a9388eb4814fbbd9d (127.14.61.3). New cstate: current_term: 1 leader_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:34.449309 19939 raft_consensus.cc:1270] T 883afaedb39b40bd94599fb96d9889fd P c0ba5bb8d237437184fc6a02ce96fa0b [term 1 FOLLOWER]: Refusing update from remote peer 0cfd85d83bd14a6a9388eb4814fbbd9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:02:34.449337 19805 raft_consensus.cc:1270] T 883afaedb39b40bd94599fb96d9889fd P c5ce5cefbeac43f5a1fe36c6dedfdf40 [term 1 FOLLOWER]: Refusing update from remote peer 0cfd85d83bd14a6a9388eb4814fbbd9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:02:34.449869 20151 consensus_queue.cc:1035] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:02:34.450030 20157 consensus_queue.cc:1035] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:02:34.452441 20150 tablet_replica.cc:440] T 883afaedb39b40bd94599fb96d9889fd P c5ce5cefbeac43f5a1fe36c6dedfdf40: TxnStatusTablet state changed. Reason: New leader 0cfd85d83bd14a6a9388eb4814fbbd9d. Latest consensus state: current_term: 1 leader_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } } }
I20250111 09:02:34.452538 20150 tablet_replica.cc:443] T 883afaedb39b40bd94599fb96d9889fd P c5ce5cefbeac43f5a1fe36c6dedfdf40: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:34.453521 20211 tablet_replica.cc:440] T 883afaedb39b40bd94599fb96d9889fd P c0ba5bb8d237437184fc6a02ce96fa0b: TxnStatusTablet state changed. Reason: New leader 0cfd85d83bd14a6a9388eb4814fbbd9d. Latest consensus state: current_term: 1 leader_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } } }
I20250111 09:02:34.453619 20211 tablet_replica.cc:443] T 883afaedb39b40bd94599fb96d9889fd P c0ba5bb8d237437184fc6a02ce96fa0b: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:34.455099 20151 tablet_replica.cc:440] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } } }
I20250111 09:02:34.455178 20151 tablet_replica.cc:443] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:02:34.455849 20151 tablet_replica.cc:440] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } } }
I20250111 09:02:34.455852 20150 tablet_replica.cc:440] T 883afaedb39b40bd94599fb96d9889fd P c5ce5cefbeac43f5a1fe36c6dedfdf40: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } } }
I20250111 09:02:34.455922 20151 tablet_replica.cc:443] T 883afaedb39b40bd94599fb96d9889fd P 0cfd85d83bd14a6a9388eb4814fbbd9d: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:02:34.455922 20150 tablet_replica.cc:443] T 883afaedb39b40bd94599fb96d9889fd P c5ce5cefbeac43f5a1fe36c6dedfdf40: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:34.458679 20211 tablet_replica.cc:440] T 883afaedb39b40bd94599fb96d9889fd P c0ba5bb8d237437184fc6a02ce96fa0b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 } } peers { permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 } } peers { permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 } } }
I20250111 09:02:34.458755 20211 tablet_replica.cc:443] T 883afaedb39b40bd94599fb96d9889fd P c0ba5bb8d237437184fc6a02ce96fa0b: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:34.459569 20216 mvcc.cc:204] Tried to move back new op lower bound from 7113056888619479040 to 7113056888401678336. Current Snapshot: MvccSnapshot[applied={T|T < 7113056888619479040 or (T in {7113056888619479040})}]
I20250111 09:02:34.459995 20218 mvcc.cc:204] Tried to move back new op lower bound from 7113056888619479040 to 7113056888401678336. Current Snapshot: MvccSnapshot[applied={T|T < 7113056888619479040}]
I20250111 09:02:34.468987 20224 consensus_queue.cc:1035] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5ce5cefbeac43f5a1fe36c6dedfdf40" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:02:34.474097 20224 consensus_queue.cc:1035] T f9c6d25609134cb58c4a74c15daa6824 P 0cfd85d83bd14a6a9388eb4814fbbd9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:02:34.500950 20150 consensus_queue.cc:1035] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cfd85d83bd14a6a9388eb4814fbbd9d" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:02:34.516644 20150 consensus_queue.cc:1035] T 203944a7a4194f69822687d24790e320 P c5ce5cefbeac43f5a1fe36c6dedfdf40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0ba5bb8d237437184fc6a02ce96fa0b" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:02:35.322975 14580 txn_write_ops-itest.cc:831] txn write RPCs completed: 20
I20250111 09:02:35.323043 14580 txn_write_ops-itest.cc:833] txn write RPC rate: 400 req/sec
I20250111 09:02:35.340121 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 19721
I20250111 09:02:35.348834 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 19858
I20250111 09:02:35.354614 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 19995
I20250111 09:02:35.361081 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 19615
2025-01-11T09:02:35Z chronyd exiting
[ OK ] TxnWriteOpsITest.WriteOpPerf (2262 ms)
[ RUN ] TxnWriteOpsITest.WriteOpForNonExistentTxn
2025-01-11T09:02:35Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-01-11T09:02:35Z Disabled control of system clock
I20250111 09:02:35.392567 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.61.62:41275
--webserver_interface=127.14.61.62
--webserver_port=0
--builtin_ntp_servers=127.14.61.20:35301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.61.62:41275
--txn_manager_enabled=true with env {}
W20250111 09:02:35.483263 20305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:02:35.483436 20305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:02:35.483497 20305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:02:35.486315 20305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250111 09:02:35.486387 20305 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20250111 09:02:35.486409 20305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:02:35.486461 20305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250111 09:02:35.486495 20305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250111 09:02:35.489571 20305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:35301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.61.62:41275
--txn_manager_enabled=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.61.62:41275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.61.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.20305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:02:35.490417 20305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:02:35.491312 20305 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:35.497527 20310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:35.498157 20313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:35.498657 20311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:35.499356 20305 server_base.cc:1034] running on GCE node
I20250111 09:02:35.499682 20305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:02:35.500425 20305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:02:35.501552 20305 hybrid_clock.cc:648] HybridClock initialized: now 1736586155501536 us; error 29 us; skew 500 ppm
I20250111 09:02:35.502945 20305 webserver.cc:458] Webserver started at http://127.14.61.62:38851/ using document root <none> and password file <none>
I20250111 09:02:35.503405 20305 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:35.503453 20305 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:35.503582 20305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:02:35.504835 20305 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/master-0/data/instance:
uuid: "6a85748181944b259573039fc7ffc102"
format_stamp: "Formatted at 2025-01-11 09:02:35 on dist-test-slave-nvbt"
I20250111 09:02:35.505264 20305 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/master-0/wal/instance:
uuid: "6a85748181944b259573039fc7ffc102"
format_stamp: "Formatted at 2025-01-11 09:02:35 on dist-test-slave-nvbt"
I20250111 09:02:35.507958 20305 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20250111 09:02:35.509624 20319 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:35.510475 20305 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:35.510558 20305 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/master-0/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/master-0/wal
uuid: "6a85748181944b259573039fc7ffc102"
format_stamp: "Formatted at 2025-01-11 09:02:35 on dist-test-slave-nvbt"
I20250111 09:02:35.510620 20305 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:35.528069 20305 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:35.528527 20305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:02:35.528646 20305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:35.535064 20305 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:41275
I20250111 09:02:35.535071 20381 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:41275 every 8 connection(s)
I20250111 09:02:35.535825 20305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
I20250111 09:02:35.536648 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 20305
I20250111 09:02:35.536742 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/master-0/wal/instance
I20250111 09:02:35.538138 20382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:35.542105 20382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102: Bootstrap starting.
I20250111 09:02:35.544075 20382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:35.544692 20382 log.cc:826] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102: Log is configured to *not* fsync() on all Append() calls
I20250111 09:02:35.546053 20382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102: No bootstrap required, opened a new log
I20250111 09:02:35.548513 20382 raft_consensus.cc:357] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a85748181944b259573039fc7ffc102" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 41275 } }
I20250111 09:02:35.548650 20382 raft_consensus.cc:383] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:35.548702 20382 raft_consensus.cc:738] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a85748181944b259573039fc7ffc102, State: Initialized, Role: FOLLOWER
I20250111 09:02:35.549060 20382 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a85748181944b259573039fc7ffc102" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 41275 } }
I20250111 09:02:35.549189 20382 raft_consensus.cc:397] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:02:35.549263 20382 raft_consensus.cc:491] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:02:35.549384 20382 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:35.550005 20382 raft_consensus.cc:513] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a85748181944b259573039fc7ffc102" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 41275 } }
I20250111 09:02:35.550256 20382 leader_election.cc:304] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a85748181944b259573039fc7ffc102; no voters:
I20250111 09:02:35.550472 20382 leader_election.cc:290] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:02:35.550568 20387 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:35.550765 20387 raft_consensus.cc:695] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102 [term 1 LEADER]: Becoming Leader. State: Replica: 6a85748181944b259573039fc7ffc102, State: Running, Role: LEADER
I20250111 09:02:35.551071 20387 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a85748181944b259573039fc7ffc102" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 41275 } }
I20250111 09:02:35.551388 20382 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102 [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:02:35.552325 20389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a85748181944b259573039fc7ffc102. Latest consensus state: current_term: 1 leader_uuid: "6a85748181944b259573039fc7ffc102" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a85748181944b259573039fc7ffc102" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 41275 } } }
I20250111 09:02:35.552452 20389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102 [sys.catalog]: This master's current role is: LEADER
I20250111 09:02:35.552770 20396 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:02:35.553305 20388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a85748181944b259573039fc7ffc102" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a85748181944b259573039fc7ffc102" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 41275 } } }
I20250111 09:02:35.553368 20388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102 [sys.catalog]: This master's current role is: LEADER
I20250111 09:02:35.555382 20396 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:02:35.559221 20396 catalog_manager.cc:1348] Generated new cluster ID: b199da1763e24e2491f33ff8a8613cf5
I20250111 09:02:35.559278 20396 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:02:35.585510 20396 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:02:35.586262 20396 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:02:35.590581 20396 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 6a85748181944b259573039fc7ffc102: Generated new TSK 0
I20250111 09:02:35.591090 20396 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:02:35.601755 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.1:0
--local_ip_for_outbound_sockets=127.14.61.1
--webserver_interface=127.14.61.1
--webserver_port=0
--tserver_master_addrs=127.14.61.62:41275
--builtin_ntp_servers=127.14.61.20:35301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_txn_system_client_init=true with env {}
W20250111 09:02:35.698190 20406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:02:35.698364 20406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:02:35.698402 20406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:02:35.701397 20406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:02:35.701453 20406 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:02:35.701495 20406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.1
I20250111 09:02:35.704504 20406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:35301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.61.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:41275
--never_fsync=true
--heap_profile_path=/tmp/kudu.20406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.1
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:02:35.705369 20406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:02:35.706297 20406 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:35.713275 20414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:35.713960 20412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:35.714660 20411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:35.715348 20406 server_base.cc:1034] running on GCE node
I20250111 09:02:35.715634 20406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:02:35.716096 20406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:02:35.717232 20406 hybrid_clock.cc:648] HybridClock initialized: now 1736586155717210 us; error 34 us; skew 500 ppm
I20250111 09:02:35.718827 20406 webserver.cc:458] Webserver started at http://127.14.61.1:39469/ using document root <none> and password file <none>
I20250111 09:02:35.719391 20406 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:35.719457 20406 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:35.719628 20406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:02:35.721381 20406 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-0/data/instance:
uuid: "f7cd33890a894e598d265c17aa3b456c"
format_stamp: "Formatted at 2025-01-11 09:02:35 on dist-test-slave-nvbt"
I20250111 09:02:35.721812 20406 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-0/wal/instance:
uuid: "f7cd33890a894e598d265c17aa3b456c"
format_stamp: "Formatted at 2025-01-11 09:02:35 on dist-test-slave-nvbt"
I20250111 09:02:35.724574 20406 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20250111 09:02:35.726480 20420 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:35.727304 20406 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20250111 09:02:35.727394 20406 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-0/wal
uuid: "f7cd33890a894e598d265c17aa3b456c"
format_stamp: "Formatted at 2025-01-11 09:02:35 on dist-test-slave-nvbt"
I20250111 09:02:35.727461 20406 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:35.744393 20406 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:35.744794 20406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:02:35.744918 20406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:35.746173 20406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:02:35.746222 20406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:35.746275 20406 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:02:35.746323 20406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:35.756141 20406 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:43443
I20250111 09:02:35.756201 20540 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:43443 every 8 connection(s)
I20250111 09:02:35.757200 20406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
I20250111 09:02:35.758363 20541 heartbeater.cc:346] Connected to a master server at 127.14.61.62:41275
I20250111 09:02:35.758474 20541 heartbeater.cc:463] Registering TS with master...
I20250111 09:02:35.758803 20541 heartbeater.cc:510] Master 127.14.61.62:41275 requested a full tablet report, sending...
I20250111 09:02:35.760061 20336 ts_manager.cc:194] Registered new tserver with Master: f7cd33890a894e598d265c17aa3b456c (127.14.61.1:43443)
I20250111 09:02:35.761196 20336 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.1:33137
I20250111 09:02:35.767292 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 20406
I20250111 09:02:35.767386 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-0/wal/instance
I20250111 09:02:35.769066 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.2:0
--local_ip_for_outbound_sockets=127.14.61.2
--webserver_interface=127.14.61.2
--webserver_port=0
--tserver_master_addrs=127.14.61.62:41275
--builtin_ntp_servers=127.14.61.20:35301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_txn_system_client_init=true with env {}
W20250111 09:02:35.862545 20543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:02:35.862705 20543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:02:35.862763 20543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:02:35.865470 20543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:02:35.865541 20543 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:02:35.865628 20543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.2
I20250111 09:02:35.868876 20543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:35301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.61.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:41275
--never_fsync=true
--heap_profile_path=/tmp/kudu.20543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.2
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:02:35.869643 20543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:02:35.870579 20543 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:35.877736 20549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:35.878379 20551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:35.878959 20548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:35.879629 20543 server_base.cc:1034] running on GCE node
I20250111 09:02:35.879925 20543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:02:35.880367 20543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:02:35.881537 20543 hybrid_clock.cc:648] HybridClock initialized: now 1736586155881511 us; error 38 us; skew 500 ppm
I20250111 09:02:35.882932 20543 webserver.cc:458] Webserver started at http://127.14.61.2:37947/ using document root <none> and password file <none>
I20250111 09:02:35.883409 20543 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:35.883462 20543 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:35.883615 20543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:02:35.884965 20543 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-1/data/instance:
uuid: "013fdf9acda94784b6542bb34aff02b2"
format_stamp: "Formatted at 2025-01-11 09:02:35 on dist-test-slave-nvbt"
I20250111 09:02:35.885463 20543 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-1/wal/instance:
uuid: "013fdf9acda94784b6542bb34aff02b2"
format_stamp: "Formatted at 2025-01-11 09:02:35 on dist-test-slave-nvbt"
I20250111 09:02:35.888200 20543 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20250111 09:02:35.889901 20557 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:35.890751 20543 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20250111 09:02:35.890852 20543 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-1/wal
uuid: "013fdf9acda94784b6542bb34aff02b2"
format_stamp: "Formatted at 2025-01-11 09:02:35 on dist-test-slave-nvbt"
I20250111 09:02:35.890924 20543 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:35.909970 20543 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:35.910440 20543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:02:35.910588 20543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:35.911723 20543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:02:35.911777 20543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:35.911818 20543 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:02:35.911844 20543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:35.921304 20543 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.2:45309
I20250111 09:02:35.921358 20677 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.2:45309 every 8 connection(s)
I20250111 09:02:35.922029 20543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
I20250111 09:02:35.923075 20678 heartbeater.cc:346] Connected to a master server at 127.14.61.62:41275
I20250111 09:02:35.923172 20678 heartbeater.cc:463] Registering TS with master...
I20250111 09:02:35.923538 20678 heartbeater.cc:510] Master 127.14.61.62:41275 requested a full tablet report, sending...
I20250111 09:02:35.924008 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 20543
I20250111 09:02:35.924109 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-1/wal/instance
I20250111 09:02:35.924316 20336 ts_manager.cc:194] Registered new tserver with Master: 013fdf9acda94784b6542bb34aff02b2 (127.14.61.2:45309)
I20250111 09:02:35.924832 20336 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.2:49103
I20250111 09:02:35.926095 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.3:0
--local_ip_for_outbound_sockets=127.14.61.3
--webserver_interface=127.14.61.3
--webserver_port=0
--tserver_master_addrs=127.14.61.62:41275
--builtin_ntp_servers=127.14.61.20:35301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_txn_system_client_init=true with env {}
W20250111 09:02:36.018038 20680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:02:36.018180 20680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:02:36.018218 20680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:02:36.021080 20680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:02:36.021135 20680 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:02:36.021168 20680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.3
I20250111 09:02:36.024065 20680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:35301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.61.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:41275
--never_fsync=true
--heap_profile_path=/tmp/kudu.20680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.3
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:02:36.024770 20680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:02:36.025725 20680 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:36.032968 20685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:36.033504 20686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:36.034093 20688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:36.034708 20680 server_base.cc:1034] running on GCE node
I20250111 09:02:36.034987 20680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:02:36.035387 20680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:02:36.036516 20680 hybrid_clock.cc:648] HybridClock initialized: now 1736586156036511 us; error 22 us; skew 500 ppm
I20250111 09:02:36.037921 20680 webserver.cc:458] Webserver started at http://127.14.61.3:35699/ using document root <none> and password file <none>
I20250111 09:02:36.038362 20680 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:36.038414 20680 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:36.038553 20680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:02:36.039898 20680 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-2/data/instance:
uuid: "ca30b27029154378bec0477e9c3a004f"
format_stamp: "Formatted at 2025-01-11 09:02:36 on dist-test-slave-nvbt"
I20250111 09:02:36.040334 20680 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-2/wal/instance:
uuid: "ca30b27029154378bec0477e9c3a004f"
format_stamp: "Formatted at 2025-01-11 09:02:36 on dist-test-slave-nvbt"
I20250111 09:02:36.043038 20680 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20250111 09:02:36.044646 20694 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:36.045533 20680 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:36.045635 20680 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-2/wal
uuid: "ca30b27029154378bec0477e9c3a004f"
format_stamp: "Formatted at 2025-01-11 09:02:36 on dist-test-slave-nvbt"
I20250111 09:02:36.045706 20680 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:36.063941 20680 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:36.064342 20680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:02:36.064460 20680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:36.065642 20680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:02:36.065698 20680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:36.065743 20680 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:02:36.065773 20680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:36.075552 20680 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.3:36161
I20250111 09:02:36.075611 20814 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.3:36161 every 8 connection(s)
I20250111 09:02:36.076309 20680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
I20250111 09:02:36.077469 20815 heartbeater.cc:346] Connected to a master server at 127.14.61.62:41275
I20250111 09:02:36.077590 20815 heartbeater.cc:463] Registering TS with master...
I20250111 09:02:36.077988 20815 heartbeater.cc:510] Master 127.14.61.62:41275 requested a full tablet report, sending...
I20250111 09:02:36.078764 20335 ts_manager.cc:194] Registered new tserver with Master: ca30b27029154378bec0477e9c3a004f (127.14.61.3:36161)
I20250111 09:02:36.079427 20335 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.3:39509
I20250111 09:02:36.080804 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 20680
I20250111 09:02:36.080905 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.WriteOpForNonExistentTxn.1736586070622223-14580-0/minicluster-data/ts-2/wal/instance
I20250111 09:02:36.082639 14580 external_mini_cluster.cc:934] 3 TS(s) registered with all masters
I20250111 09:02:36.089507 20335 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56162:
name: "txn_write_ops_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
W20250111 09:02:36.091883 20335 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table txn_write_ops_test in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250111 09:02:36.105765 20607 tablet_service.cc:1467] Processing CreateTablet for tablet 41e3daaea68740ddb37300d8668761a3 (DEFAULT_TABLE table=txn_write_ops_test [id=60c2f0aa37394bad9d98804469fa3c99]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:36.105763 20743 tablet_service.cc:1467] Processing CreateTablet for tablet c25cfc7c64174eb59e674d3cab153d9c (DEFAULT_TABLE table=txn_write_ops_test [id=60c2f0aa37394bad9d98804469fa3c99]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:36.105763 20469 tablet_service.cc:1467] Processing CreateTablet for tablet c25cfc7c64174eb59e674d3cab153d9c (DEFAULT_TABLE table=txn_write_ops_test [id=60c2f0aa37394bad9d98804469fa3c99]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:36.106544 20607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41e3daaea68740ddb37300d8668761a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:36.106577 20469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c25cfc7c64174eb59e674d3cab153d9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:36.106577 20743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c25cfc7c64174eb59e674d3cab153d9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:36.107656 20744 tablet_service.cc:1467] Processing CreateTablet for tablet 41e3daaea68740ddb37300d8668761a3 (DEFAULT_TABLE table=txn_write_ops_test [id=60c2f0aa37394bad9d98804469fa3c99]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:36.108665 20744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41e3daaea68740ddb37300d8668761a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:36.109261 20606 tablet_service.cc:1467] Processing CreateTablet for tablet c25cfc7c64174eb59e674d3cab153d9c (DEFAULT_TABLE table=txn_write_ops_test [id=60c2f0aa37394bad9d98804469fa3c99]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:36.109520 20606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c25cfc7c64174eb59e674d3cab153d9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:36.105765 20470 tablet_service.cc:1467] Processing CreateTablet for tablet 41e3daaea68740ddb37300d8668761a3 (DEFAULT_TABLE table=txn_write_ops_test [id=60c2f0aa37394bad9d98804469fa3c99]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:36.111110 20470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41e3daaea68740ddb37300d8668761a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:36.113140 20832 tablet_bootstrap.cc:492] T 41e3daaea68740ddb37300d8668761a3 P ca30b27029154378bec0477e9c3a004f: Bootstrap starting.
I20250111 09:02:36.114223 20833 tablet_bootstrap.cc:492] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c: Bootstrap starting.
I20250111 09:02:36.115586 20832 tablet_bootstrap.cc:654] T 41e3daaea68740ddb37300d8668761a3 P ca30b27029154378bec0477e9c3a004f: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:36.116274 20832 log.cc:826] T 41e3daaea68740ddb37300d8668761a3 P ca30b27029154378bec0477e9c3a004f: Log is configured to *not* fsync() on all Append() calls
I20250111 09:02:36.116418 20833 tablet_bootstrap.cc:654] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:36.117063 20833 log.cc:826] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c: Log is configured to *not* fsync() on all Append() calls
I20250111 09:02:36.117255 20831 tablet_bootstrap.cc:492] T 41e3daaea68740ddb37300d8668761a3 P 013fdf9acda94784b6542bb34aff02b2: Bootstrap starting.
I20250111 09:02:36.117661 20832 tablet_bootstrap.cc:492] T 41e3daaea68740ddb37300d8668761a3 P ca30b27029154378bec0477e9c3a004f: No bootstrap required, opened a new log
I20250111 09:02:36.117832 20832 ts_tablet_manager.cc:1397] T 41e3daaea68740ddb37300d8668761a3 P ca30b27029154378bec0477e9c3a004f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20250111 09:02:36.118470 20833 tablet_bootstrap.cc:492] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c: No bootstrap required, opened a new log
I20250111 09:02:36.118631 20833 ts_tablet_manager.cc:1397] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20250111 09:02:36.119166 20831 tablet_bootstrap.cc:654] T 41e3daaea68740ddb37300d8668761a3 P 013fdf9acda94784b6542bb34aff02b2: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:36.119836 20831 log.cc:826] T 41e3daaea68740ddb37300d8668761a3 P 013fdf9acda94784b6542bb34aff02b2: Log is configured to *not* fsync() on all Append() calls
I20250111 09:02:36.120945 20832 raft_consensus.cc:357] T 41e3daaea68740ddb37300d8668761a3 P ca30b27029154378bec0477e9c3a004f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } } peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } }
I20250111 09:02:36.121181 20832 raft_consensus.cc:383] T 41e3daaea68740ddb37300d8668761a3 P ca30b27029154378bec0477e9c3a004f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:36.121248 20831 tablet_bootstrap.cc:492] T 41e3daaea68740ddb37300d8668761a3 P 013fdf9acda94784b6542bb34aff02b2: No bootstrap required, opened a new log
I20250111 09:02:36.121246 20832 raft_consensus.cc:738] T 41e3daaea68740ddb37300d8668761a3 P ca30b27029154378bec0477e9c3a004f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca30b27029154378bec0477e9c3a004f, State: Initialized, Role: FOLLOWER
I20250111 09:02:36.121400 20831 ts_tablet_manager.cc:1397] T 41e3daaea68740ddb37300d8668761a3 P 013fdf9acda94784b6542bb34aff02b2: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20250111 09:02:36.121650 20832 consensus_queue.cc:260] T 41e3daaea68740ddb37300d8668761a3 P ca30b27029154378bec0477e9c3a004f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } } peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } }
I20250111 09:02:36.121878 20833 raft_consensus.cc:357] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } } peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } }
I20250111 09:02:36.122051 20833 raft_consensus.cc:383] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:36.122102 20833 raft_consensus.cc:738] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7cd33890a894e598d265c17aa3b456c, State: Initialized, Role: FOLLOWER
I20250111 09:02:36.122306 20815 heartbeater.cc:502] Master 127.14.61.62:41275 was elected leader, sending a full tablet report...
I20250111 09:02:36.122464 20833 consensus_queue.cc:260] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } } peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } }
I20250111 09:02:36.122994 20541 heartbeater.cc:502] Master 127.14.61.62:41275 was elected leader, sending a full tablet report...
I20250111 09:02:36.123208 20833 ts_tablet_manager.cc:1428] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20250111 09:02:36.123441 20833 tablet_bootstrap.cc:492] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c: Bootstrap starting.
I20250111 09:02:36.123867 20831 raft_consensus.cc:357] T 41e3daaea68740ddb37300d8668761a3 P 013fdf9acda94784b6542bb34aff02b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } } peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } }
I20250111 09:02:36.124022 20831 raft_consensus.cc:383] T 41e3daaea68740ddb37300d8668761a3 P 013fdf9acda94784b6542bb34aff02b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:36.124071 20831 raft_consensus.cc:738] T 41e3daaea68740ddb37300d8668761a3 P 013fdf9acda94784b6542bb34aff02b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 013fdf9acda94784b6542bb34aff02b2, State: Initialized, Role: FOLLOWER
I20250111 09:02:36.124370 20833 tablet_bootstrap.cc:654] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:36.124372 20831 consensus_queue.cc:260] T 41e3daaea68740ddb37300d8668761a3 P 013fdf9acda94784b6542bb34aff02b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } } peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } }
I20250111 09:02:36.124943 20832 ts_tablet_manager.cc:1428] T 41e3daaea68740ddb37300d8668761a3 P ca30b27029154378bec0477e9c3a004f: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250111 09:02:36.125119 20831 ts_tablet_manager.cc:1428] T 41e3daaea68740ddb37300d8668761a3 P 013fdf9acda94784b6542bb34aff02b2: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20250111 09:02:36.125201 20832 tablet_bootstrap.cc:492] T c25cfc7c64174eb59e674d3cab153d9c P ca30b27029154378bec0477e9c3a004f: Bootstrap starting.
I20250111 09:02:36.126111 20678 heartbeater.cc:502] Master 127.14.61.62:41275 was elected leader, sending a full tablet report...
I20250111 09:02:36.127317 20833 tablet_bootstrap.cc:492] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c: No bootstrap required, opened a new log
I20250111 09:02:36.127393 20833 ts_tablet_manager.cc:1397] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20250111 09:02:36.127424 20832 tablet_bootstrap.cc:654] T c25cfc7c64174eb59e674d3cab153d9c P ca30b27029154378bec0477e9c3a004f: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:36.127810 20833 raft_consensus.cc:357] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } }
I20250111 09:02:36.127902 20833 raft_consensus.cc:383] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:36.127930 20833 raft_consensus.cc:738] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7cd33890a894e598d265c17aa3b456c, State: Initialized, Role: FOLLOWER
I20250111 09:02:36.128053 20833 consensus_queue.cc:260] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } }
I20250111 09:02:36.128306 20832 tablet_bootstrap.cc:492] T c25cfc7c64174eb59e674d3cab153d9c P ca30b27029154378bec0477e9c3a004f: No bootstrap required, opened a new log
I20250111 09:02:36.128352 20833 ts_tablet_manager.cc:1428] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:02:36.128377 20832 ts_tablet_manager.cc:1397] T c25cfc7c64174eb59e674d3cab153d9c P ca30b27029154378bec0477e9c3a004f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250111 09:02:36.128728 20831 tablet_bootstrap.cc:492] T c25cfc7c64174eb59e674d3cab153d9c P 013fdf9acda94784b6542bb34aff02b2: Bootstrap starting.
I20250111 09:02:36.128820 20832 raft_consensus.cc:357] T c25cfc7c64174eb59e674d3cab153d9c P ca30b27029154378bec0477e9c3a004f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } }
I20250111 09:02:36.128904 20832 raft_consensus.cc:383] T c25cfc7c64174eb59e674d3cab153d9c P ca30b27029154378bec0477e9c3a004f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:36.128937 20832 raft_consensus.cc:738] T c25cfc7c64174eb59e674d3cab153d9c P ca30b27029154378bec0477e9c3a004f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca30b27029154378bec0477e9c3a004f, State: Initialized, Role: FOLLOWER
I20250111 09:02:36.129084 20832 consensus_queue.cc:260] T c25cfc7c64174eb59e674d3cab153d9c P ca30b27029154378bec0477e9c3a004f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } }
I20250111 09:02:36.129638 20832 ts_tablet_manager.cc:1428] T c25cfc7c64174eb59e674d3cab153d9c P ca30b27029154378bec0477e9c3a004f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:02:36.129717 20831 tablet_bootstrap.cc:654] T c25cfc7c64174eb59e674d3cab153d9c P 013fdf9acda94784b6542bb34aff02b2: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:36.130841 20831 tablet_bootstrap.cc:492] T c25cfc7c64174eb59e674d3cab153d9c P 013fdf9acda94784b6542bb34aff02b2: No bootstrap required, opened a new log
I20250111 09:02:36.130936 20831 ts_tablet_manager.cc:1397] T c25cfc7c64174eb59e674d3cab153d9c P 013fdf9acda94784b6542bb34aff02b2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:02:36.131309 20831 raft_consensus.cc:357] T c25cfc7c64174eb59e674d3cab153d9c P 013fdf9acda94784b6542bb34aff02b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } }
I20250111 09:02:36.131386 20831 raft_consensus.cc:383] T c25cfc7c64174eb59e674d3cab153d9c P 013fdf9acda94784b6542bb34aff02b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:36.131425 20831 raft_consensus.cc:738] T c25cfc7c64174eb59e674d3cab153d9c P 013fdf9acda94784b6542bb34aff02b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 013fdf9acda94784b6542bb34aff02b2, State: Initialized, Role: FOLLOWER
I20250111 09:02:36.131560 20831 consensus_queue.cc:260] T c25cfc7c64174eb59e674d3cab153d9c P 013fdf9acda94784b6542bb34aff02b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } }
I20250111 09:02:36.131788 20831 ts_tablet_manager.cc:1428] T c25cfc7c64174eb59e674d3cab153d9c P 013fdf9acda94784b6542bb34aff02b2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20250111 09:02:36.172955 20679 tablet.cc:2367] T 41e3daaea68740ddb37300d8668761a3 P 013fdf9acda94784b6542bb34aff02b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250111 09:02:36.186385 20838 raft_consensus.cc:491] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:02:36.186496 20838 raft_consensus.cc:513] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } }
I20250111 09:02:36.187417 20838 leader_election.cc:290] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 013fdf9acda94784b6542bb34aff02b2 (127.14.61.2:45309), ca30b27029154378bec0477e9c3a004f (127.14.61.3:36161)
I20250111 09:02:36.191330 20761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c25cfc7c64174eb59e674d3cab153d9c" candidate_uuid: "f7cd33890a894e598d265c17aa3b456c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca30b27029154378bec0477e9c3a004f" is_pre_election: true
I20250111 09:02:36.191329 20627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c25cfc7c64174eb59e674d3cab153d9c" candidate_uuid: "f7cd33890a894e598d265c17aa3b456c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "013fdf9acda94784b6542bb34aff02b2" is_pre_election: true
I20250111 09:02:36.191546 20627 raft_consensus.cc:2463] T c25cfc7c64174eb59e674d3cab153d9c P 013fdf9acda94784b6542bb34aff02b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7cd33890a894e598d265c17aa3b456c in term 0.
I20250111 09:02:36.191537 20761 raft_consensus.cc:2463] T c25cfc7c64174eb59e674d3cab153d9c P ca30b27029154378bec0477e9c3a004f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7cd33890a894e598d265c17aa3b456c in term 0.
I20250111 09:02:36.191967 20424 leader_election.cc:304] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca30b27029154378bec0477e9c3a004f, f7cd33890a894e598d265c17aa3b456c; no voters:
I20250111 09:02:36.192173 20838 raft_consensus.cc:2798] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:02:36.192250 20838 raft_consensus.cc:491] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:02:36.192310 20838 raft_consensus.cc:3054] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:36.193033 20838 raft_consensus.cc:513] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } }
I20250111 09:02:36.193297 20838 leader_election.cc:290] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c [CANDIDATE]: Term 1 election: Requested vote from peers 013fdf9acda94784b6542bb34aff02b2 (127.14.61.2:45309), ca30b27029154378bec0477e9c3a004f (127.14.61.3:36161)
I20250111 09:02:36.193588 20761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c25cfc7c64174eb59e674d3cab153d9c" candidate_uuid: "f7cd33890a894e598d265c17aa3b456c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca30b27029154378bec0477e9c3a004f"
I20250111 09:02:36.193593 20627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c25cfc7c64174eb59e674d3cab153d9c" candidate_uuid: "f7cd33890a894e598d265c17aa3b456c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "013fdf9acda94784b6542bb34aff02b2"
I20250111 09:02:36.193678 20627 raft_consensus.cc:3054] T c25cfc7c64174eb59e674d3cab153d9c P 013fdf9acda94784b6542bb34aff02b2 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:36.193696 20761 raft_consensus.cc:3054] T c25cfc7c64174eb59e674d3cab153d9c P ca30b27029154378bec0477e9c3a004f [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:36.194386 20627 raft_consensus.cc:2463] T c25cfc7c64174eb59e674d3cab153d9c P 013fdf9acda94784b6542bb34aff02b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7cd33890a894e598d265c17aa3b456c in term 1.
I20250111 09:02:36.194399 20761 raft_consensus.cc:2463] T c25cfc7c64174eb59e674d3cab153d9c P ca30b27029154378bec0477e9c3a004f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7cd33890a894e598d265c17aa3b456c in term 1.
I20250111 09:02:36.194605 20424 leader_election.cc:304] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca30b27029154378bec0477e9c3a004f, f7cd33890a894e598d265c17aa3b456c; no voters:
I20250111 09:02:36.194774 20838 raft_consensus.cc:2798] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:36.194937 20838 raft_consensus.cc:695] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c [term 1 LEADER]: Becoming Leader. State: Replica: f7cd33890a894e598d265c17aa3b456c, State: Running, Role: LEADER
I20250111 09:02:36.195199 20838 consensus_queue.cc:237] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } }
I20250111 09:02:36.197532 20336 catalog_manager.cc:5526] T c25cfc7c64174eb59e674d3cab153d9c P f7cd33890a894e598d265c17aa3b456c reported cstate change: term changed from 0 to 1, leader changed from <none> to f7cd33890a894e598d265c17aa3b456c (127.14.61.1). New cstate: current_term: 1 leader_uuid: "f7cd33890a894e598d265c17aa3b456c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:36.199874 20838 raft_consensus.cc:491] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:02:36.199949 20838 raft_consensus.cc:513] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } } peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } }
I20250111 09:02:36.200213 20838 leader_election.cc:290] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 013fdf9acda94784b6542bb34aff02b2 (127.14.61.2:45309), ca30b27029154378bec0477e9c3a004f (127.14.61.3:36161)
I20250111 09:02:36.200457 20627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e3daaea68740ddb37300d8668761a3" candidate_uuid: "f7cd33890a894e598d265c17aa3b456c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "013fdf9acda94784b6542bb34aff02b2" is_pre_election: true
I20250111 09:02:36.200549 20761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e3daaea68740ddb37300d8668761a3" candidate_uuid: "f7cd33890a894e598d265c17aa3b456c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca30b27029154378bec0477e9c3a004f" is_pre_election: true
I20250111 09:02:36.200569 20627 raft_consensus.cc:2463] T 41e3daaea68740ddb37300d8668761a3 P 013fdf9acda94784b6542bb34aff02b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7cd33890a894e598d265c17aa3b456c in term 0.
I20250111 09:02:36.200654 20761 raft_consensus.cc:2463] T 41e3daaea68740ddb37300d8668761a3 P ca30b27029154378bec0477e9c3a004f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7cd33890a894e598d265c17aa3b456c in term 0.
I20250111 09:02:36.200851 20423 leader_election.cc:304] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 013fdf9acda94784b6542bb34aff02b2, f7cd33890a894e598d265c17aa3b456c; no voters:
I20250111 09:02:36.201059 20838 raft_consensus.cc:2798] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:02:36.201103 20838 raft_consensus.cc:491] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:02:36.201130 20838 raft_consensus.cc:3054] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:36.201638 20838 raft_consensus.cc:513] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } } peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } }
I20250111 09:02:36.201851 20838 leader_election.cc:290] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c [CANDIDATE]: Term 1 election: Requested vote from peers 013fdf9acda94784b6542bb34aff02b2 (127.14.61.2:45309), ca30b27029154378bec0477e9c3a004f (127.14.61.3:36161)
I20250111 09:02:36.202101 20627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e3daaea68740ddb37300d8668761a3" candidate_uuid: "f7cd33890a894e598d265c17aa3b456c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "013fdf9acda94784b6542bb34aff02b2"
I20250111 09:02:36.202152 20761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e3daaea68740ddb37300d8668761a3" candidate_uuid: "f7cd33890a894e598d265c17aa3b456c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca30b27029154378bec0477e9c3a004f"
I20250111 09:02:36.202191 20627 raft_consensus.cc:3054] T 41e3daaea68740ddb37300d8668761a3 P 013fdf9acda94784b6542bb34aff02b2 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:36.202224 20761 raft_consensus.cc:3054] T 41e3daaea68740ddb37300d8668761a3 P ca30b27029154378bec0477e9c3a004f [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:36.202800 20627 raft_consensus.cc:2463] T 41e3daaea68740ddb37300d8668761a3 P 013fdf9acda94784b6542bb34aff02b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7cd33890a894e598d265c17aa3b456c in term 1.
I20250111 09:02:36.202836 20761 raft_consensus.cc:2463] T 41e3daaea68740ddb37300d8668761a3 P ca30b27029154378bec0477e9c3a004f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7cd33890a894e598d265c17aa3b456c in term 1.
I20250111 09:02:36.203089 20423 leader_election.cc:304] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 013fdf9acda94784b6542bb34aff02b2, f7cd33890a894e598d265c17aa3b456c; no voters:
I20250111 09:02:36.203249 20838 raft_consensus.cc:2798] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:36.203325 20838 raft_consensus.cc:695] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c [term 1 LEADER]: Becoming Leader. State: Replica: f7cd33890a894e598d265c17aa3b456c, State: Running, Role: LEADER
I20250111 09:02:36.203469 20838 consensus_queue.cc:237] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } } peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } }
I20250111 09:02:36.204634 20336 catalog_manager.cc:5526] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c reported cstate change: term changed from 0 to 1, leader changed from <none> to f7cd33890a894e598d265c17aa3b456c (127.14.61.1). New cstate: current_term: 1 leader_uuid: "f7cd33890a894e598d265c17aa3b456c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } health_report { overall_health: HEALTHY } } }
W20250111 09:02:36.258195 20542 tablet.cc:2367] T 41e3daaea68740ddb37300d8668761a3 P f7cd33890a894e598d265c17aa3b456c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250111 09:02:36.264200 20336 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56168:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20250111 09:02:36.264705 20336 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250111 09:02:36.268517 20607 tablet_service.cc:1467] Processing CreateTablet for tablet 73352c8699c246d59c77a799d74425c3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e533512224ac45e1b5b15938ed3cd5fe]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:02:36.268450 20469 tablet_service.cc:1467] Processing CreateTablet for tablet 73352c8699c246d59c77a799d74425c3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e533512224ac45e1b5b15938ed3cd5fe]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:02:36.268662 20743 tablet_service.cc:1467] Processing CreateTablet for tablet 73352c8699c246d59c77a799d74425c3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e533512224ac45e1b5b15938ed3cd5fe]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:02:36.268988 20469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73352c8699c246d59c77a799d74425c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:36.268990 20607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73352c8699c246d59c77a799d74425c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:36.269048 20743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73352c8699c246d59c77a799d74425c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:36.270880 20831 tablet_bootstrap.cc:492] T 73352c8699c246d59c77a799d74425c3 P 013fdf9acda94784b6542bb34aff02b2: Bootstrap starting.
I20250111 09:02:36.271302 20832 tablet_bootstrap.cc:492] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f: Bootstrap starting.
I20250111 09:02:36.271884 20831 tablet_bootstrap.cc:654] T 73352c8699c246d59c77a799d74425c3 P 013fdf9acda94784b6542bb34aff02b2: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:36.272277 20832 tablet_bootstrap.cc:654] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:36.272617 20833 tablet_bootstrap.cc:492] T 73352c8699c246d59c77a799d74425c3 P f7cd33890a894e598d265c17aa3b456c: Bootstrap starting.
I20250111 09:02:36.272959 20831 tablet_bootstrap.cc:492] T 73352c8699c246d59c77a799d74425c3 P 013fdf9acda94784b6542bb34aff02b2: No bootstrap required, opened a new log
I20250111 09:02:36.273056 20831 ts_tablet_manager.cc:1397] T 73352c8699c246d59c77a799d74425c3 P 013fdf9acda94784b6542bb34aff02b2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:36.273412 20832 tablet_bootstrap.cc:492] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f: No bootstrap required, opened a new log
I20250111 09:02:36.273495 20832 ts_tablet_manager.cc:1397] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:36.273530 20833 tablet_bootstrap.cc:654] T 73352c8699c246d59c77a799d74425c3 P f7cd33890a894e598d265c17aa3b456c: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:36.273515 20831 raft_consensus.cc:357] T 73352c8699c246d59c77a799d74425c3 P 013fdf9acda94784b6542bb34aff02b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } }
I20250111 09:02:36.273597 20831 raft_consensus.cc:383] T 73352c8699c246d59c77a799d74425c3 P 013fdf9acda94784b6542bb34aff02b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:36.273627 20831 raft_consensus.cc:738] T 73352c8699c246d59c77a799d74425c3 P 013fdf9acda94784b6542bb34aff02b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 013fdf9acda94784b6542bb34aff02b2, State: Initialized, Role: FOLLOWER
I20250111 09:02:36.273730 20831 consensus_queue.cc:260] T 73352c8699c246d59c77a799d74425c3 P 013fdf9acda94784b6542bb34aff02b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } }
I20250111 09:02:36.273908 20832 raft_consensus.cc:357] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } }
I20250111 09:02:36.273998 20832 raft_consensus.cc:383] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:36.274025 20832 raft_consensus.cc:738] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca30b27029154378bec0477e9c3a004f, State: Initialized, Role: FOLLOWER
I20250111 09:02:36.273960 20839 tablet_replica.cc:440] T 73352c8699c246d59c77a799d74425c3 P 013fdf9acda94784b6542bb34aff02b2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } } }
I20250111 09:02:36.274084 20839 tablet_replica.cc:443] T 73352c8699c246d59c77a799d74425c3 P 013fdf9acda94784b6542bb34aff02b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:36.274139 20832 consensus_queue.cc:260] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } }
I20250111 09:02:36.274403 20832 ts_tablet_manager.cc:1428] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:02:36.274375 20837 tablet_replica.cc:440] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } } }
I20250111 09:02:36.274485 20837 tablet_replica.cc:443] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:36.273960 20831 ts_tablet_manager.cc:1428] T 73352c8699c246d59c77a799d74425c3 P 013fdf9acda94784b6542bb34aff02b2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:02:36.275288 20833 tablet_bootstrap.cc:492] T 73352c8699c246d59c77a799d74425c3 P f7cd33890a894e598d265c17aa3b456c: No bootstrap required, opened a new log
I20250111 09:02:36.275373 20833 ts_tablet_manager.cc:1397] T 73352c8699c246d59c77a799d74425c3 P f7cd33890a894e598d265c17aa3b456c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250111 09:02:36.275816 20833 raft_consensus.cc:357] T 73352c8699c246d59c77a799d74425c3 P f7cd33890a894e598d265c17aa3b456c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } }
I20250111 09:02:36.275885 20833 raft_consensus.cc:383] T 73352c8699c246d59c77a799d74425c3 P f7cd33890a894e598d265c17aa3b456c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:36.275908 20833 raft_consensus.cc:738] T 73352c8699c246d59c77a799d74425c3 P f7cd33890a894e598d265c17aa3b456c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7cd33890a894e598d265c17aa3b456c, State: Initialized, Role: FOLLOWER
I20250111 09:02:36.276021 20833 consensus_queue.cc:260] T 73352c8699c246d59c77a799d74425c3 P f7cd33890a894e598d265c17aa3b456c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } }
I20250111 09:02:36.276283 20833 ts_tablet_manager.cc:1428] T 73352c8699c246d59c77a799d74425c3 P f7cd33890a894e598d265c17aa3b456c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250111 09:02:36.276262 20838 tablet_replica.cc:440] T 73352c8699c246d59c77a799d74425c3 P f7cd33890a894e598d265c17aa3b456c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } } }
I20250111 09:02:36.276362 20838 tablet_replica.cc:443] T 73352c8699c246d59c77a799d74425c3 P f7cd33890a894e598d265c17aa3b456c: This TxnStatusTablet replica's current role is: FOLLOWER
W20250111 09:02:36.327425 20816 tablet.cc:2367] T 41e3daaea68740ddb37300d8668761a3 P ca30b27029154378bec0477e9c3a004f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250111 09:02:36.436321 20837 raft_consensus.cc:491] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:02:36.436436 20837 raft_consensus.cc:513] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } }
I20250111 09:02:36.437518 20837 leader_election.cc:290] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7cd33890a894e598d265c17aa3b456c (127.14.61.1:43443), 013fdf9acda94784b6542bb34aff02b2 (127.14.61.2:45309)
I20250111 09:02:36.441337 20627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73352c8699c246d59c77a799d74425c3" candidate_uuid: "ca30b27029154378bec0477e9c3a004f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "013fdf9acda94784b6542bb34aff02b2" is_pre_election: true
I20250111 09:02:36.441459 20627 raft_consensus.cc:2463] T 73352c8699c246d59c77a799d74425c3 P 013fdf9acda94784b6542bb34aff02b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca30b27029154378bec0477e9c3a004f in term 0.
I20250111 09:02:36.441466 20490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73352c8699c246d59c77a799d74425c3" candidate_uuid: "ca30b27029154378bec0477e9c3a004f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7cd33890a894e598d265c17aa3b456c" is_pre_election: true
I20250111 09:02:36.441675 20490 raft_consensus.cc:2463] T 73352c8699c246d59c77a799d74425c3 P f7cd33890a894e598d265c17aa3b456c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca30b27029154378bec0477e9c3a004f in term 0.
I20250111 09:02:36.441730 20697 leader_election.cc:304] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 013fdf9acda94784b6542bb34aff02b2, ca30b27029154378bec0477e9c3a004f; no voters:
I20250111 09:02:36.441932 20837 raft_consensus.cc:2798] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:02:36.441987 20837 raft_consensus.cc:491] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:02:36.442013 20837 raft_consensus.cc:3054] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:36.442745 20837 raft_consensus.cc:513] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } }
I20250111 09:02:36.443033 20837 leader_election.cc:290] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f [CANDIDATE]: Term 1 election: Requested vote from peers f7cd33890a894e598d265c17aa3b456c (127.14.61.1:43443), 013fdf9acda94784b6542bb34aff02b2 (127.14.61.2:45309)
I20250111 09:02:36.443298 20490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73352c8699c246d59c77a799d74425c3" candidate_uuid: "ca30b27029154378bec0477e9c3a004f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7cd33890a894e598d265c17aa3b456c"
I20250111 09:02:36.443382 20490 raft_consensus.cc:3054] T 73352c8699c246d59c77a799d74425c3 P f7cd33890a894e598d265c17aa3b456c [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:36.443336 20627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73352c8699c246d59c77a799d74425c3" candidate_uuid: "ca30b27029154378bec0477e9c3a004f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "013fdf9acda94784b6542bb34aff02b2"
I20250111 09:02:36.443463 20627 raft_consensus.cc:3054] T 73352c8699c246d59c77a799d74425c3 P 013fdf9acda94784b6542bb34aff02b2 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:36.444103 20490 raft_consensus.cc:2463] T 73352c8699c246d59c77a799d74425c3 P f7cd33890a894e598d265c17aa3b456c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca30b27029154378bec0477e9c3a004f in term 1.
I20250111 09:02:36.444125 20627 raft_consensus.cc:2463] T 73352c8699c246d59c77a799d74425c3 P 013fdf9acda94784b6542bb34aff02b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca30b27029154378bec0477e9c3a004f in term 1.
I20250111 09:02:36.444363 20697 leader_election.cc:304] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 013fdf9acda94784b6542bb34aff02b2, ca30b27029154378bec0477e9c3a004f; no voters:
I20250111 09:02:36.444522 20837 raft_consensus.cc:2798] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:36.444671 20837 raft_consensus.cc:695] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f [term 1 LEADER]: Becoming Leader. State: Replica: ca30b27029154378bec0477e9c3a004f, State: Running, Role: LEADER
I20250111 09:02:36.444911 20837 consensus_queue.cc:237] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } }
I20250111 09:02:36.446952 20858 tablet_replica.cc:440] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f: TxnStatusTablet state changed. Reason: New leader ca30b27029154378bec0477e9c3a004f. Latest consensus state: current_term: 1 leader_uuid: "ca30b27029154378bec0477e9c3a004f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } } }
I20250111 09:02:36.447042 20858 tablet_replica.cc:443] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:02:36.447274 20860 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:02:36.447440 20860 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:02:36.447868 20335 catalog_manager.cc:5526] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f reported cstate change: term changed from 0 to 1, leader changed from <none> to ca30b27029154378bec0477e9c3a004f (127.14.61.3). New cstate: current_term: 1 leader_uuid: "ca30b27029154378bec0477e9c3a004f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:36.466495 20627 raft_consensus.cc:1270] T 73352c8699c246d59c77a799d74425c3 P 013fdf9acda94784b6542bb34aff02b2 [term 1 FOLLOWER]: Refusing update from remote peer ca30b27029154378bec0477e9c3a004f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:02:36.466493 20490 raft_consensus.cc:1270] T 73352c8699c246d59c77a799d74425c3 P f7cd33890a894e598d265c17aa3b456c [term 1 FOLLOWER]: Refusing update from remote peer ca30b27029154378bec0477e9c3a004f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:02:36.466964 20837 consensus_queue.cc:1035] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:02:36.467157 20858 consensus_queue.cc:1035] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f [LEADER]: Connected to new peer: Peer: permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:02:36.469859 20838 tablet_replica.cc:440] T 73352c8699c246d59c77a799d74425c3 P f7cd33890a894e598d265c17aa3b456c: TxnStatusTablet state changed. Reason: New leader ca30b27029154378bec0477e9c3a004f. Latest consensus state: current_term: 1 leader_uuid: "ca30b27029154378bec0477e9c3a004f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } } }
I20250111 09:02:36.469957 20838 tablet_replica.cc:443] T 73352c8699c246d59c77a799d74425c3 P f7cd33890a894e598d265c17aa3b456c: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:36.470917 20839 tablet_replica.cc:440] T 73352c8699c246d59c77a799d74425c3 P 013fdf9acda94784b6542bb34aff02b2: TxnStatusTablet state changed. Reason: New leader ca30b27029154378bec0477e9c3a004f. Latest consensus state: current_term: 1 leader_uuid: "ca30b27029154378bec0477e9c3a004f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } } }
I20250111 09:02:36.471031 20839 tablet_replica.cc:443] T 73352c8699c246d59c77a799d74425c3 P 013fdf9acda94784b6542bb34aff02b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:36.472591 20858 tablet_replica.cc:440] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca30b27029154378bec0477e9c3a004f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } } }
I20250111 09:02:36.472668 20858 tablet_replica.cc:443] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:02:36.473122 20837 tablet_replica.cc:440] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca30b27029154378bec0477e9c3a004f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } } }
I20250111 09:02:36.473201 20837 tablet_replica.cc:443] T 73352c8699c246d59c77a799d74425c3 P ca30b27029154378bec0477e9c3a004f: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:02:36.473802 20865 mvcc.cc:204] Tried to move back new op lower bound from 7113056896881479680 to 7113056896800522240. Current Snapshot: MvccSnapshot[applied={T|T < 7113056896881479680}]
I20250111 09:02:36.474020 20838 tablet_replica.cc:440] T 73352c8699c246d59c77a799d74425c3 P f7cd33890a894e598d265c17aa3b456c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca30b27029154378bec0477e9c3a004f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } } }
I20250111 09:02:36.474092 20838 tablet_replica.cc:443] T 73352c8699c246d59c77a799d74425c3 P f7cd33890a894e598d265c17aa3b456c: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:36.475731 20839 tablet_replica.cc:440] T 73352c8699c246d59c77a799d74425c3 P 013fdf9acda94784b6542bb34aff02b2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca30b27029154378bec0477e9c3a004f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cd33890a894e598d265c17aa3b456c" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43443 } } peers { permanent_uuid: "013fdf9acda94784b6542bb34aff02b2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45309 } } peers { permanent_uuid: "ca30b27029154378bec0477e9c3a004f" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 36161 } } }
I20250111 09:02:36.475821 20839 tablet_replica.cc:443] T 73352c8699c246d59c77a799d74425c3 P 013fdf9acda94784b6542bb34aff02b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:36.477288 20864 mvcc.cc:204] Tried to move back new op lower bound from 7113056896881479680 to 7113056896800522240. Current Snapshot: MvccSnapshot[applied={T|T < 7113056896881479680 or (T in {7113056896881479680})}]
W20250111 09:02:36.490813 20872 tablet_replica.cc:1307] Invalid argument: Failed to write to server: ca30b27029154378bec0477e9c3a004f (127.14.61.3:36161): transaction ID 100 not found, current highest txn ID: 0: cancelling pending write operations
W20250111 09:02:36.490929 20872 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:48406: Invalid argument: Failed to write to server: ca30b27029154378bec0477e9c3a004f (127.14.61.3:36161): transaction ID 100 not found, current highest txn ID: 0
W20250111 09:02:36.491544 20818 batcher.cc:441] Invalid argument: Failed to write batch of 1 ops to tablet c25cfc7c64174eb59e674d3cab153d9c after 1 attempt(s): Failed to write to server: f7cd33890a894e598d265c17aa3b456c (127.14.61.1:43443): Failed to write to server: ca30b27029154378bec0477e9c3a004f (127.14.61.3:36161): transaction ID 100 not found, current highest txn ID: 0
I20250111 09:02:36.491775 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 20406
I20250111 09:02:36.498400 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 20543
I20250111 09:02:36.505450 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 20680
I20250111 09:02:36.509963 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 20305
2025-01-11T09:02:36Z chronyd exiting
[ OK ] TxnWriteOpsITest.WriteOpForNonExistentTxn (1146 ms)
[ RUN ] TxnWriteOpsITest.TxnWriteAfterCommit
2025-01-11T09:02:36Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-01-11T09:02:36Z Disabled control of system clock
I20250111 09:02:36.537549 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.61.62:43479
--webserver_interface=127.14.61.62
--webserver_port=0
--builtin_ntp_servers=127.14.61.20:44415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.61.62:43479
--txn_manager_enabled=true with env {}
W20250111 09:02:36.625599 20878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:02:36.625763 20878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:02:36.625799 20878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:02:36.628476 20878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250111 09:02:36.628517 20878 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20250111 09:02:36.628531 20878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:02:36.628546 20878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250111 09:02:36.628568 20878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250111 09:02:36.631840 20878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:44415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.61.62:43479
--txn_manager_enabled=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.61.62:43479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.61.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.20878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:02:36.632565 20878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:02:36.633396 20878 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:36.639353 20883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:36.640022 20884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:36.640755 20886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:36.641400 20878 server_base.cc:1034] running on GCE node
I20250111 09:02:36.641748 20878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:02:36.642438 20878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:02:36.643586 20878 hybrid_clock.cc:648] HybridClock initialized: now 1736586156643564 us; error 29 us; skew 500 ppm
I20250111 09:02:36.645067 20878 webserver.cc:458] Webserver started at http://127.14.61.62:38253/ using document root <none> and password file <none>
I20250111 09:02:36.645581 20878 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:36.645639 20878 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:36.645793 20878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:02:36.647104 20878 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/master-0/data/instance:
uuid: "be7fae6678c946fc99166098c7b4e3f7"
format_stamp: "Formatted at 2025-01-11 09:02:36 on dist-test-slave-nvbt"
I20250111 09:02:36.647480 20878 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/master-0/wal/instance:
uuid: "be7fae6678c946fc99166098c7b4e3f7"
format_stamp: "Formatted at 2025-01-11 09:02:36 on dist-test-slave-nvbt"
I20250111 09:02:36.650151 20878 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20250111 09:02:36.651961 20892 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:36.652889 20878 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:36.652989 20878 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/master-0/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/master-0/wal
uuid: "be7fae6678c946fc99166098c7b4e3f7"
format_stamp: "Formatted at 2025-01-11 09:02:36 on dist-test-slave-nvbt"
I20250111 09:02:36.653098 20878 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:36.662633 20878 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:36.663072 20878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:02:36.663218 20878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:36.669891 20878 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:43479
I20250111 09:02:36.669900 20954 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:43479 every 8 connection(s)
I20250111 09:02:36.670748 20878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/master-0/data/info.pb
I20250111 09:02:36.671137 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 20878
I20250111 09:02:36.671238 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/master-0/wal/instance
I20250111 09:02:36.673164 20955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:36.677197 20955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7: Bootstrap starting.
I20250111 09:02:36.679162 20955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:36.679734 20955 log.cc:826] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7: Log is configured to *not* fsync() on all Append() calls
I20250111 09:02:36.681115 20955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7: No bootstrap required, opened a new log
I20250111 09:02:36.683645 20955 raft_consensus.cc:357] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be7fae6678c946fc99166098c7b4e3f7" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 43479 } }
I20250111 09:02:36.683809 20955 raft_consensus.cc:383] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:36.683866 20955 raft_consensus.cc:738] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be7fae6678c946fc99166098c7b4e3f7, State: Initialized, Role: FOLLOWER
I20250111 09:02:36.684235 20955 consensus_queue.cc:260] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be7fae6678c946fc99166098c7b4e3f7" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 43479 } }
I20250111 09:02:36.684341 20955 raft_consensus.cc:397] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:02:36.684397 20955 raft_consensus.cc:491] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:02:36.684490 20955 raft_consensus.cc:3054] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:36.685101 20955 raft_consensus.cc:513] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be7fae6678c946fc99166098c7b4e3f7" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 43479 } }
I20250111 09:02:36.685418 20955 leader_election.cc:304] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be7fae6678c946fc99166098c7b4e3f7; no voters:
I20250111 09:02:36.685638 20955 leader_election.cc:290] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:02:36.685740 20960 raft_consensus.cc:2798] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:36.685914 20960 raft_consensus.cc:695] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7 [term 1 LEADER]: Becoming Leader. State: Replica: be7fae6678c946fc99166098c7b4e3f7, State: Running, Role: LEADER
I20250111 09:02:36.686177 20960 consensus_queue.cc:237] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be7fae6678c946fc99166098c7b4e3f7" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 43479 } }
I20250111 09:02:36.686656 20955 sys_catalog.cc:564] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7 [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:02:36.687276 20962 sys_catalog.cc:455] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be7fae6678c946fc99166098c7b4e3f7. Latest consensus state: current_term: 1 leader_uuid: "be7fae6678c946fc99166098c7b4e3f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be7fae6678c946fc99166098c7b4e3f7" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 43479 } } }
I20250111 09:02:36.687394 20962 sys_catalog.cc:458] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7 [sys.catalog]: This master's current role is: LEADER
I20250111 09:02:36.687620 20961 sys_catalog.cc:455] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be7fae6678c946fc99166098c7b4e3f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be7fae6678c946fc99166098c7b4e3f7" member_type: VOTER last_known_addr { host: "127.14.61.62" port: 43479 } } }
I20250111 09:02:36.687711 20961 sys_catalog.cc:458] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7 [sys.catalog]: This master's current role is: LEADER
I20250111 09:02:36.687731 20968 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:02:36.690065 20968 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:02:36.694450 20968 catalog_manager.cc:1348] Generated new cluster ID: db5ee06b9c1444a686c2459408ff2700
I20250111 09:02:36.694505 20968 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:02:36.711913 20968 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:02:36.712791 20968 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:02:36.720690 20968 catalog_manager.cc:5899] T 00000000000000000000000000000000 P be7fae6678c946fc99166098c7b4e3f7: Generated new TSK 0
I20250111 09:02:36.721315 20968 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:02:36.730858 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.1:0
--local_ip_for_outbound_sockets=127.14.61.1
--webserver_interface=127.14.61.1
--webserver_port=0
--tserver_master_addrs=127.14.61.62:43479
--builtin_ntp_servers=127.14.61.20:44415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_txn_system_client_init=true with env {}
W20250111 09:02:36.824083 20979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:02:36.824241 20979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:02:36.824278 20979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:02:36.827071 20979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:02:36.827136 20979 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:02:36.827183 20979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.1
I20250111 09:02:36.830386 20979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:44415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.61.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:43479
--never_fsync=true
--heap_profile_path=/tmp/kudu.20979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.1
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:02:36.831225 20979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:02:36.832185 20979 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:36.839253 20985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:36.839824 20987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:36.840525 20984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:36.841300 20979 server_base.cc:1034] running on GCE node
I20250111 09:02:36.841575 20979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:02:36.841993 20979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:02:36.843140 20979 hybrid_clock.cc:648] HybridClock initialized: now 1736586156843123 us; error 35 us; skew 500 ppm
I20250111 09:02:36.844599 20979 webserver.cc:458] Webserver started at http://127.14.61.1:41241/ using document root <none> and password file <none>
I20250111 09:02:36.845069 20979 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:36.845137 20979 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:36.845345 20979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:02:36.846771 20979 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/data/instance:
uuid: "f7ac20146503466cad420c7364808411"
format_stamp: "Formatted at 2025-01-11 09:02:36 on dist-test-slave-nvbt"
I20250111 09:02:36.847194 20979 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/wal/instance:
uuid: "f7ac20146503466cad420c7364808411"
format_stamp: "Formatted at 2025-01-11 09:02:36 on dist-test-slave-nvbt"
I20250111 09:02:36.849908 20979 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20250111 09:02:36.851748 20993 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:36.852560 20979 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:36.852648 20979 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/wal
uuid: "f7ac20146503466cad420c7364808411"
format_stamp: "Formatted at 2025-01-11 09:02:36 on dist-test-slave-nvbt"
I20250111 09:02:36.852712 20979 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:36.877640 20979 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:36.878026 20979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:02:36.878145 20979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:36.879333 20979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:02:36.879379 20979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:36.879431 20979 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:02:36.879460 20979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:36.889302 20979 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:36483
I20250111 09:02:36.889343 21113 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:36483 every 8 connection(s)
I20250111 09:02:36.890053 20979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
I20250111 09:02:36.891582 21114 heartbeater.cc:346] Connected to a master server at 127.14.61.62:43479
I20250111 09:02:36.891695 21114 heartbeater.cc:463] Registering TS with master...
I20250111 09:02:36.892055 21114 heartbeater.cc:510] Master 127.14.61.62:43479 requested a full tablet report, sending...
I20250111 09:02:36.893153 20908 ts_manager.cc:194] Registered new tserver with Master: f7ac20146503466cad420c7364808411 (127.14.61.1:36483)
I20250111 09:02:36.894105 20908 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.1:36255
I20250111 09:02:36.896111 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 20979
I20250111 09:02:36.896231 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/wal/instance
I20250111 09:02:36.897927 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.2:0
--local_ip_for_outbound_sockets=127.14.61.2
--webserver_interface=127.14.61.2
--webserver_port=0
--tserver_master_addrs=127.14.61.62:43479
--builtin_ntp_servers=127.14.61.20:44415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_txn_system_client_init=true with env {}
W20250111 09:02:36.988173 21116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:02:36.988361 21116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:02:36.988427 21116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:02:36.991425 21116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:02:36.991492 21116 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:02:36.991555 21116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.2
I20250111 09:02:36.994621 21116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:44415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.61.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:43479
--never_fsync=true
--heap_profile_path=/tmp/kudu.21116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.2
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:02:36.995460 21116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:02:36.996367 21116 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:37.003696 21124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:37.004402 21122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:37.005070 21121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:37.005811 21116 server_base.cc:1034] running on GCE node
I20250111 09:02:37.006083 21116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:02:37.006521 21116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:02:37.007652 21116 hybrid_clock.cc:648] HybridClock initialized: now 1736586157007642 us; error 23 us; skew 500 ppm
I20250111 09:02:37.009131 21116 webserver.cc:458] Webserver started at http://127.14.61.2:44933/ using document root <none> and password file <none>
I20250111 09:02:37.009572 21116 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:37.009615 21116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:37.009760 21116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:02:37.011206 21116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/data/instance:
uuid: "741496baa99c4e189c56a345852c2ea2"
format_stamp: "Formatted at 2025-01-11 09:02:37 on dist-test-slave-nvbt"
I20250111 09:02:37.011613 21116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/wal/instance:
uuid: "741496baa99c4e189c56a345852c2ea2"
format_stamp: "Formatted at 2025-01-11 09:02:37 on dist-test-slave-nvbt"
I20250111 09:02:37.014250 21116 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.005s sys 0.000s
I20250111 09:02:37.015944 21130 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:37.016763 21116 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:37.016847 21116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/wal
uuid: "741496baa99c4e189c56a345852c2ea2"
format_stamp: "Formatted at 2025-01-11 09:02:37 on dist-test-slave-nvbt"
I20250111 09:02:37.016911 21116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:37.032682 21116 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:37.033095 21116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:02:37.033201 21116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:37.034224 21116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:02:37.034268 21116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:37.034327 21116 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:02:37.034350 21116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:37.043246 21116 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.2:44379
I20250111 09:02:37.043301 21249 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.2:44379 every 8 connection(s)
I20250111 09:02:37.044093 21116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
I20250111 09:02:37.045261 21251 heartbeater.cc:346] Connected to a master server at 127.14.61.62:43479
I20250111 09:02:37.045377 21251 heartbeater.cc:463] Registering TS with master...
I20250111 09:02:37.045785 21251 heartbeater.cc:510] Master 127.14.61.62:43479 requested a full tablet report, sending...
I20250111 09:02:37.046701 20908 ts_manager.cc:194] Registered new tserver with Master: 741496baa99c4e189c56a345852c2ea2 (127.14.61.2:44379)
I20250111 09:02:37.047262 20908 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.2:41285
I20250111 09:02:37.052675 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 21116
I20250111 09:02:37.052788 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/wal/instance
I20250111 09:02:37.054454 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.3:0
--local_ip_for_outbound_sockets=127.14.61.3
--webserver_interface=127.14.61.3
--webserver_port=0
--tserver_master_addrs=127.14.61.62:43479
--builtin_ntp_servers=127.14.61.20:44415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_txn_system_client_init=true with env {}
W20250111 09:02:37.144446 21253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:02:37.144598 21253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:02:37.144652 21253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:02:37.147430 21253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:02:37.147493 21253 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:02:37.147557 21253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.3
I20250111 09:02:37.150821 21253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:44415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.61.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:43479
--never_fsync=true
--heap_profile_path=/tmp/kudu.21253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.3
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:02:37.151611 21253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:02:37.152467 21253 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:37.159292 21258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:37.160035 21259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:37.160642 21261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:37.161378 21253 server_base.cc:1034] running on GCE node
I20250111 09:02:37.161666 21253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:02:37.162074 21253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:02:37.163209 21253 hybrid_clock.cc:648] HybridClock initialized: now 1736586157163201 us; error 30 us; skew 500 ppm
I20250111 09:02:37.164669 21253 webserver.cc:458] Webserver started at http://127.14.61.3:35389/ using document root <none> and password file <none>
I20250111 09:02:37.165184 21253 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:37.165244 21253 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:37.165393 21253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:02:37.166764 21253 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/data/instance:
uuid: "921d1c0189ff47bfa769e489f9d4ef11"
format_stamp: "Formatted at 2025-01-11 09:02:37 on dist-test-slave-nvbt"
I20250111 09:02:37.167166 21253 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/wal/instance:
uuid: "921d1c0189ff47bfa769e489f9d4ef11"
format_stamp: "Formatted at 2025-01-11 09:02:37 on dist-test-slave-nvbt"
I20250111 09:02:37.169924 21253 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20250111 09:02:37.171639 21267 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:37.172349 21253 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20250111 09:02:37.172442 21253 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/wal
uuid: "921d1c0189ff47bfa769e489f9d4ef11"
format_stamp: "Formatted at 2025-01-11 09:02:37 on dist-test-slave-nvbt"
I20250111 09:02:37.172504 21253 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:37.183957 21253 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:37.184357 21253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:02:37.184474 21253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:37.185668 21253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:02:37.185727 21253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:37.185782 21253 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:02:37.185817 21253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:37.195051 21253 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.3:44997
I20250111 09:02:37.195089 21387 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.3:44997 every 8 connection(s)
I20250111 09:02:37.195780 21253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
I20250111 09:02:37.196885 21388 heartbeater.cc:346] Connected to a master server at 127.14.61.62:43479
I20250111 09:02:37.196987 21388 heartbeater.cc:463] Registering TS with master...
I20250111 09:02:37.197412 21388 heartbeater.cc:510] Master 127.14.61.62:43479 requested a full tablet report, sending...
I20250111 09:02:37.198107 20909 ts_manager.cc:194] Registered new tserver with Master: 921d1c0189ff47bfa769e489f9d4ef11 (127.14.61.3:44997)
I20250111 09:02:37.198628 20909 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.3:58321
I20250111 09:02:37.199108 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 21253
I20250111 09:02:37.199209 14580 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/wal/instance
I20250111 09:02:37.201220 14580 external_mini_cluster.cc:934] 3 TS(s) registered with all masters
I20250111 09:02:37.207855 20909 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54712:
name: "txn_write_ops_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
W20250111 09:02:37.210163 20909 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table txn_write_ops_test in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250111 09:02:37.223717 21317 tablet_service.cc:1467] Processing CreateTablet for tablet 470943c2f9784db0a50bb8d4c207c1a3 (DEFAULT_TABLE table=txn_write_ops_test [id=68ae10652a904fc988d9a87e59297471]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:37.223718 21180 tablet_service.cc:1467] Processing CreateTablet for tablet 470943c2f9784db0a50bb8d4c207c1a3 (DEFAULT_TABLE table=txn_write_ops_test [id=68ae10652a904fc988d9a87e59297471]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:37.223762 21043 tablet_service.cc:1467] Processing CreateTablet for tablet 470943c2f9784db0a50bb8d4c207c1a3 (DEFAULT_TABLE table=txn_write_ops_test [id=68ae10652a904fc988d9a87e59297471]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:37.224133 21179 tablet_service.cc:1467] Processing CreateTablet for tablet 1858ad7ec2ca435eb404436d8159656a (DEFAULT_TABLE table=txn_write_ops_test [id=68ae10652a904fc988d9a87e59297471]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:37.223762 21042 tablet_service.cc:1467] Processing CreateTablet for tablet 1858ad7ec2ca435eb404436d8159656a (DEFAULT_TABLE table=txn_write_ops_test [id=68ae10652a904fc988d9a87e59297471]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:37.224529 21180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 470943c2f9784db0a50bb8d4c207c1a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:37.224529 21317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 470943c2f9784db0a50bb8d4c207c1a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:37.224581 21043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 470943c2f9784db0a50bb8d4c207c1a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:37.224923 21179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1858ad7ec2ca435eb404436d8159656a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:37.225875 21042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1858ad7ec2ca435eb404436d8159656a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:37.226678 21316 tablet_service.cc:1467] Processing CreateTablet for tablet 1858ad7ec2ca435eb404436d8159656a (DEFAULT_TABLE table=txn_write_ops_test [id=68ae10652a904fc988d9a87e59297471]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:37.226897 21316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1858ad7ec2ca435eb404436d8159656a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:37.230783 21401 tablet_bootstrap.cc:492] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11: Bootstrap starting.
I20250111 09:02:37.231094 21402 tablet_bootstrap.cc:492] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411: Bootstrap starting.
I20250111 09:02:37.231447 21403 tablet_bootstrap.cc:492] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2: Bootstrap starting.
I20250111 09:02:37.232904 21401 tablet_bootstrap.cc:654] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:37.233335 21402 tablet_bootstrap.cc:654] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:37.233489 21403 tablet_bootstrap.cc:654] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:37.233628 21401 log.cc:826] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11: Log is configured to *not* fsync() on all Append() calls
I20250111 09:02:37.233989 21402 log.cc:826] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411: Log is configured to *not* fsync() on all Append() calls
I20250111 09:02:37.234030 21403 log.cc:826] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2: Log is configured to *not* fsync() on all Append() calls
I20250111 09:02:37.234951 21401 tablet_bootstrap.cc:492] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11: No bootstrap required, opened a new log
I20250111 09:02:37.235129 21401 ts_tablet_manager.cc:1397] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20250111 09:02:37.235272 21403 tablet_bootstrap.cc:492] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2: No bootstrap required, opened a new log
I20250111 09:02:37.235347 21402 tablet_bootstrap.cc:492] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411: No bootstrap required, opened a new log
I20250111 09:02:37.235441 21403 ts_tablet_manager.cc:1397] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20250111 09:02:37.235513 21402 ts_tablet_manager.cc:1397] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20250111 09:02:37.237603 21403 raft_consensus.cc:357] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.237741 21403 raft_consensus.cc:383] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:37.237788 21403 raft_consensus.cc:738] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 741496baa99c4e189c56a345852c2ea2, State: Initialized, Role: FOLLOWER
I20250111 09:02:37.237881 21401 raft_consensus.cc:357] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.238049 21401 raft_consensus.cc:383] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:37.238106 21401 raft_consensus.cc:738] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921d1c0189ff47bfa769e489f9d4ef11, State: Initialized, Role: FOLLOWER
I20250111 09:02:37.238093 21402 raft_consensus.cc:357] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.238164 21403 consensus_queue.cc:260] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.238248 21402 raft_consensus.cc:383] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:37.238314 21402 raft_consensus.cc:738] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ac20146503466cad420c7364808411, State: Initialized, Role: FOLLOWER
I20250111 09:02:37.238516 21401 consensus_queue.cc:260] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.238837 21251 heartbeater.cc:502] Master 127.14.61.62:43479 was elected leader, sending a full tablet report...
I20250111 09:02:37.238935 21403 ts_tablet_manager.cc:1428] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20250111 09:02:37.239143 21403 tablet_bootstrap.cc:492] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2: Bootstrap starting.
I20250111 09:02:37.239403 21402 consensus_queue.cc:260] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.240041 21403 tablet_bootstrap.cc:654] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:37.240234 21402 ts_tablet_manager.cc:1428] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250111 09:02:37.240556 21114 heartbeater.cc:502] Master 127.14.61.62:43479 was elected leader, sending a full tablet report...
I20250111 09:02:37.241513 21402 tablet_bootstrap.cc:492] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411: Bootstrap starting.
I20250111 09:02:37.242583 21402 tablet_bootstrap.cc:654] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:37.243638 21401 ts_tablet_manager.cc:1428] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20250111 09:02:37.243695 21402 tablet_bootstrap.cc:492] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411: No bootstrap required, opened a new log
I20250111 09:02:37.243753 21402 ts_tablet_manager.cc:1397] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:37.243925 21388 heartbeater.cc:502] Master 127.14.61.62:43479 was elected leader, sending a full tablet report...
I20250111 09:02:37.244243 21402 raft_consensus.cc:357] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } }
I20250111 09:02:37.244341 21402 raft_consensus.cc:383] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:37.244375 21402 raft_consensus.cc:738] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ac20146503466cad420c7364808411, State: Initialized, Role: FOLLOWER
I20250111 09:02:37.244470 21402 consensus_queue.cc:260] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } }
I20250111 09:02:37.244722 21401 tablet_bootstrap.cc:492] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11: Bootstrap starting.
I20250111 09:02:37.244776 21402 ts_tablet_manager.cc:1428] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:02:37.245783 21401 tablet_bootstrap.cc:654] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:37.247005 21401 tablet_bootstrap.cc:492] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11: No bootstrap required, opened a new log
I20250111 09:02:37.247166 21401 ts_tablet_manager.cc:1397] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:37.247243 21403 tablet_bootstrap.cc:492] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2: No bootstrap required, opened a new log
I20250111 09:02:37.247349 21403 ts_tablet_manager.cc:1397] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20250111 09:02:37.247918 21401 raft_consensus.cc:357] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } }
I20250111 09:02:37.248091 21403 raft_consensus.cc:357] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } }
I20250111 09:02:37.248109 21401 raft_consensus.cc:383] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:37.248181 21403 raft_consensus.cc:383] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:37.248220 21403 raft_consensus.cc:738] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 741496baa99c4e189c56a345852c2ea2, State: Initialized, Role: FOLLOWER
I20250111 09:02:37.248265 21401 raft_consensus.cc:738] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921d1c0189ff47bfa769e489f9d4ef11, State: Initialized, Role: FOLLOWER
I20250111 09:02:37.248347 21403 consensus_queue.cc:260] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } }
I20250111 09:02:37.248564 21403 ts_tablet_manager.cc:1428] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20250111 09:02:37.248495 21401 consensus_queue.cc:260] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } }
I20250111 09:02:37.248751 21401 ts_tablet_manager.cc:1428] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:02:37.249686 21408 raft_consensus.cc:491] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:02:37.249816 21408 raft_consensus.cc:513] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.250892 21408 leader_election.cc:290] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7ac20146503466cad420c7364808411 (127.14.61.1:36483), 741496baa99c4e189c56a345852c2ea2 (127.14.61.2:44379)
I20250111 09:02:37.254091 21054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1858ad7ec2ca435eb404436d8159656a" candidate_uuid: "921d1c0189ff47bfa769e489f9d4ef11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7ac20146503466cad420c7364808411" is_pre_election: true
I20250111 09:02:37.254292 21054 raft_consensus.cc:2463] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921d1c0189ff47bfa769e489f9d4ef11 in term 0.
I20250111 09:02:37.254343 21200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1858ad7ec2ca435eb404436d8159656a" candidate_uuid: "921d1c0189ff47bfa769e489f9d4ef11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "741496baa99c4e189c56a345852c2ea2" is_pre_election: true
I20250111 09:02:37.254536 21200 raft_consensus.cc:2463] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921d1c0189ff47bfa769e489f9d4ef11 in term 0.
I20250111 09:02:37.254637 21269 leader_election.cc:304] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 921d1c0189ff47bfa769e489f9d4ef11, f7ac20146503466cad420c7364808411; no voters:
I20250111 09:02:37.254840 21408 raft_consensus.cc:2798] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:02:37.254917 21408 raft_consensus.cc:491] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:02:37.254967 21408 raft_consensus.cc:3054] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:37.255613 21408 raft_consensus.cc:513] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.255856 21408 leader_election.cc:290] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [CANDIDATE]: Term 1 election: Requested vote from peers f7ac20146503466cad420c7364808411 (127.14.61.1:36483), 741496baa99c4e189c56a345852c2ea2 (127.14.61.2:44379)
I20250111 09:02:37.256181 21200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1858ad7ec2ca435eb404436d8159656a" candidate_uuid: "921d1c0189ff47bfa769e489f9d4ef11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "741496baa99c4e189c56a345852c2ea2"
I20250111 09:02:37.256296 21200 raft_consensus.cc:3054] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:37.257105 21054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1858ad7ec2ca435eb404436d8159656a" candidate_uuid: "921d1c0189ff47bfa769e489f9d4ef11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7ac20146503466cad420c7364808411"
I20250111 09:02:37.257112 21200 raft_consensus.cc:2463] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921d1c0189ff47bfa769e489f9d4ef11 in term 1.
I20250111 09:02:37.257189 21054 raft_consensus.cc:3054] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:37.257339 21271 leader_election.cc:304] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 741496baa99c4e189c56a345852c2ea2, 921d1c0189ff47bfa769e489f9d4ef11; no voters:
I20250111 09:02:37.257472 21408 raft_consensus.cc:2798] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:37.257629 21408 raft_consensus.cc:695] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [term 1 LEADER]: Becoming Leader. State: Replica: 921d1c0189ff47bfa769e489f9d4ef11, State: Running, Role: LEADER
I20250111 09:02:37.257848 21408 consensus_queue.cc:237] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.257926 21054 raft_consensus.cc:2463] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921d1c0189ff47bfa769e489f9d4ef11 in term 1.
I20250111 09:02:37.260001 20907 catalog_manager.cc:5526] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 reported cstate change: term changed from 0 to 1, leader changed from <none> to 921d1c0189ff47bfa769e489f9d4ef11 (127.14.61.3). New cstate: current_term: 1 leader_uuid: "921d1c0189ff47bfa769e489f9d4ef11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } health_report { overall_health: HEALTHY } } }
W20250111 09:02:37.294791 21252 tablet.cc:2367] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250111 09:02:37.391036 21115 tablet.cc:2367] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250111 09:02:37.446542 21389 tablet.cc:2367] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250111 09:02:37.473296 21407 raft_consensus.cc:491] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:02:37.473413 21407 raft_consensus.cc:513] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } }
I20250111 09:02:37.474377 21407 leader_election.cc:290] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921d1c0189ff47bfa769e489f9d4ef11 (127.14.61.3:44997), f7ac20146503466cad420c7364808411 (127.14.61.1:36483)
I20250111 09:02:37.477877 21054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "470943c2f9784db0a50bb8d4c207c1a3" candidate_uuid: "741496baa99c4e189c56a345852c2ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7ac20146503466cad420c7364808411" is_pre_election: true
I20250111 09:02:37.477965 21337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "470943c2f9784db0a50bb8d4c207c1a3" candidate_uuid: "741496baa99c4e189c56a345852c2ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921d1c0189ff47bfa769e489f9d4ef11" is_pre_election: true
I20250111 09:02:37.478008 21054 raft_consensus.cc:2463] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 741496baa99c4e189c56a345852c2ea2 in term 0.
I20250111 09:02:37.478142 21337 raft_consensus.cc:2463] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 741496baa99c4e189c56a345852c2ea2 in term 0.
I20250111 09:02:37.478294 21132 leader_election.cc:304] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 741496baa99c4e189c56a345852c2ea2, f7ac20146503466cad420c7364808411; no voters:
I20250111 09:02:37.478520 21407 raft_consensus.cc:2798] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:02:37.478588 21407 raft_consensus.cc:491] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:02:37.478629 21407 raft_consensus.cc:3054] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:37.479377 21407 raft_consensus.cc:513] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } }
I20250111 09:02:37.479646 21407 leader_election.cc:290] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [CANDIDATE]: Term 1 election: Requested vote from peers 921d1c0189ff47bfa769e489f9d4ef11 (127.14.61.3:44997), f7ac20146503466cad420c7364808411 (127.14.61.1:36483)
I20250111 09:02:37.479954 21337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "470943c2f9784db0a50bb8d4c207c1a3" candidate_uuid: "741496baa99c4e189c56a345852c2ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921d1c0189ff47bfa769e489f9d4ef11"
I20250111 09:02:37.479964 21054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "470943c2f9784db0a50bb8d4c207c1a3" candidate_uuid: "741496baa99c4e189c56a345852c2ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7ac20146503466cad420c7364808411"
I20250111 09:02:37.480062 21054 raft_consensus.cc:3054] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:37.480072 21337 raft_consensus.cc:3054] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:37.480808 21337 raft_consensus.cc:2463] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 741496baa99c4e189c56a345852c2ea2 in term 1.
I20250111 09:02:37.480823 21054 raft_consensus.cc:2463] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 741496baa99c4e189c56a345852c2ea2 in term 1.
I20250111 09:02:37.481117 21132 leader_election.cc:304] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 741496baa99c4e189c56a345852c2ea2, f7ac20146503466cad420c7364808411; no voters:
I20250111 09:02:37.481315 21407 raft_consensus.cc:2798] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:37.481472 21407 raft_consensus.cc:695] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [term 1 LEADER]: Becoming Leader. State: Replica: 741496baa99c4e189c56a345852c2ea2, State: Running, Role: LEADER
I20250111 09:02:37.481839 21407 consensus_queue.cc:237] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } }
I20250111 09:02:37.484534 20907 catalog_manager.cc:5526] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 741496baa99c4e189c56a345852c2ea2 (127.14.61.2). New cstate: current_term: 1 leader_uuid: "741496baa99c4e189c56a345852c2ea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:37.543748 20907 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54720:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20250111 09:02:37.544406 20907 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250111 09:02:37.548183 21179 tablet_service.cc:1467] Processing CreateTablet for tablet 20ccc398fb2f4029b2c7acc60252dfab (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8767e45b0e1a4186a965c4871ac987cd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:02:37.548195 21043 tablet_service.cc:1467] Processing CreateTablet for tablet 20ccc398fb2f4029b2c7acc60252dfab (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8767e45b0e1a4186a965c4871ac987cd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:02:37.548722 21043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ccc398fb2f4029b2c7acc60252dfab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:37.548722 21179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ccc398fb2f4029b2c7acc60252dfab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:37.551122 21403 tablet_bootstrap.cc:492] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2: Bootstrap starting.
I20250111 09:02:37.551448 21317 tablet_service.cc:1467] Processing CreateTablet for tablet 20ccc398fb2f4029b2c7acc60252dfab (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8767e45b0e1a4186a965c4871ac987cd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:02:37.551849 21403 tablet_bootstrap.cc:654] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:37.551877 21317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ccc398fb2f4029b2c7acc60252dfab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:37.552291 21402 tablet_bootstrap.cc:492] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411: Bootstrap starting.
I20250111 09:02:37.552783 21403 tablet_bootstrap.cc:492] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2: No bootstrap required, opened a new log
I20250111 09:02:37.552843 21403 ts_tablet_manager.cc:1397] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20250111 09:02:37.553120 21402 tablet_bootstrap.cc:654] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:37.553320 21403 raft_consensus.cc:357] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.553404 21403 raft_consensus.cc:383] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:37.553435 21403 raft_consensus.cc:738] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 741496baa99c4e189c56a345852c2ea2, State: Initialized, Role: FOLLOWER
I20250111 09:02:37.553555 21403 consensus_queue.cc:260] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.553828 21403 ts_tablet_manager.cc:1428] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:02:37.554003 21407 tablet_replica.cc:440] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } }
I20250111 09:02:37.554080 21401 tablet_bootstrap.cc:492] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11: Bootstrap starting.
I20250111 09:02:37.554100 21407 tablet_replica.cc:443] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:37.554173 21402 tablet_bootstrap.cc:492] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411: No bootstrap required, opened a new log
I20250111 09:02:37.554239 21402 ts_tablet_manager.cc:1397] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:02:37.554697 21402 raft_consensus.cc:357] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.554808 21402 raft_consensus.cc:383] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:37.554850 21402 raft_consensus.cc:738] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ac20146503466cad420c7364808411, State: Initialized, Role: FOLLOWER
I20250111 09:02:37.554986 21402 consensus_queue.cc:260] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.555045 21401 tablet_bootstrap.cc:654] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:37.555289 21402 ts_tablet_manager.cc:1428] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250111 09:02:37.555951 21412 tablet_replica.cc:440] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } }
I20250111 09:02:37.556095 21412 tablet_replica.cc:443] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:37.556631 21401 tablet_bootstrap.cc:492] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11: No bootstrap required, opened a new log
I20250111 09:02:37.556707 21401 ts_tablet_manager.cc:1397] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250111 09:02:37.557152 21401 raft_consensus.cc:357] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.557242 21401 raft_consensus.cc:383] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:37.557281 21401 raft_consensus.cc:738] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921d1c0189ff47bfa769e489f9d4ef11, State: Initialized, Role: FOLLOWER
I20250111 09:02:37.557404 21401 consensus_queue.cc:260] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.557628 21401 ts_tablet_manager.cc:1428] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:02:37.557802 21408 tablet_replica.cc:440] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } }
I20250111 09:02:37.557888 21408 tablet_replica.cc:443] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:37.610757 21407 raft_consensus.cc:491] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:02:37.610860 21407 raft_consensus.cc:513] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.611171 21407 leader_election.cc:290] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7ac20146503466cad420c7364808411 (127.14.61.1:36483), 921d1c0189ff47bfa769e489f9d4ef11 (127.14.61.3:44997)
I20250111 09:02:37.611562 21054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ccc398fb2f4029b2c7acc60252dfab" candidate_uuid: "741496baa99c4e189c56a345852c2ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7ac20146503466cad420c7364808411" is_pre_election: true
I20250111 09:02:37.611563 21337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ccc398fb2f4029b2c7acc60252dfab" candidate_uuid: "741496baa99c4e189c56a345852c2ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921d1c0189ff47bfa769e489f9d4ef11" is_pre_election: true
I20250111 09:02:37.611681 21337 raft_consensus.cc:2463] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 741496baa99c4e189c56a345852c2ea2 in term 0.
I20250111 09:02:37.611683 21054 raft_consensus.cc:2463] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 741496baa99c4e189c56a345852c2ea2 in term 0.
I20250111 09:02:37.611943 21134 leader_election.cc:304] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 741496baa99c4e189c56a345852c2ea2, 921d1c0189ff47bfa769e489f9d4ef11; no voters:
I20250111 09:02:37.612113 21407 raft_consensus.cc:2798] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:02:37.612170 21407 raft_consensus.cc:491] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:02:37.612213 21407 raft_consensus.cc:3054] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:37.612890 21407 raft_consensus.cc:513] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.613183 21407 leader_election.cc:290] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [CANDIDATE]: Term 1 election: Requested vote from peers f7ac20146503466cad420c7364808411 (127.14.61.1:36483), 921d1c0189ff47bfa769e489f9d4ef11 (127.14.61.3:44997)
I20250111 09:02:37.613497 21337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ccc398fb2f4029b2c7acc60252dfab" candidate_uuid: "741496baa99c4e189c56a345852c2ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921d1c0189ff47bfa769e489f9d4ef11"
I20250111 09:02:37.613523 21054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ccc398fb2f4029b2c7acc60252dfab" candidate_uuid: "741496baa99c4e189c56a345852c2ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7ac20146503466cad420c7364808411"
I20250111 09:02:37.613610 21337 raft_consensus.cc:3054] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:37.613615 21054 raft_consensus.cc:3054] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:37.614465 21337 raft_consensus.cc:2463] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 741496baa99c4e189c56a345852c2ea2 in term 1.
I20250111 09:02:37.614465 21054 raft_consensus.cc:2463] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 741496baa99c4e189c56a345852c2ea2 in term 1.
I20250111 09:02:37.614732 21134 leader_election.cc:304] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 741496baa99c4e189c56a345852c2ea2, 921d1c0189ff47bfa769e489f9d4ef11; no voters:
I20250111 09:02:37.614868 21407 raft_consensus.cc:2798] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:37.614961 21407 raft_consensus.cc:695] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [term 1 LEADER]: Becoming Leader. State: Replica: 741496baa99c4e189c56a345852c2ea2, State: Running, Role: LEADER
I20250111 09:02:37.615085 21407 consensus_queue.cc:237] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.615664 21418 tablet_replica.cc:440] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2: TxnStatusTablet state changed. Reason: New leader 741496baa99c4e189c56a345852c2ea2. Latest consensus state: current_term: 1 leader_uuid: "741496baa99c4e189c56a345852c2ea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } }
I20250111 09:02:37.615749 21418 tablet_replica.cc:443] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:02:37.615965 21431 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:02:37.616178 21431 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:02:37.616559 20909 catalog_manager.cc:5526] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 741496baa99c4e189c56a345852c2ea2 (127.14.61.2). New cstate: current_term: 1 leader_uuid: "741496baa99c4e189c56a345852c2ea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:37.637360 21054 raft_consensus.cc:1270] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Refusing update from remote peer 741496baa99c4e189c56a345852c2ea2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:02:37.637357 21337 raft_consensus.cc:1270] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11 [term 1 FOLLOWER]: Refusing update from remote peer 741496baa99c4e189c56a345852c2ea2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:02:37.637825 21418 consensus_queue.cc:1035] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:02:37.637959 21407 consensus_queue.cc:1035] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:02:37.640472 21408 tablet_replica.cc:440] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11: TxnStatusTablet state changed. Reason: New leader 741496baa99c4e189c56a345852c2ea2. Latest consensus state: current_term: 1 leader_uuid: "741496baa99c4e189c56a345852c2ea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } }
I20250111 09:02:37.640571 21408 tablet_replica.cc:443] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:37.641335 21412 tablet_replica.cc:440] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411: TxnStatusTablet state changed. Reason: New leader 741496baa99c4e189c56a345852c2ea2. Latest consensus state: current_term: 1 leader_uuid: "741496baa99c4e189c56a345852c2ea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } }
I20250111 09:02:37.641422 21412 tablet_replica.cc:443] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:37.642550 21434 mvcc.cc:204] Tried to move back new op lower bound from 7113056901677805568 to 7113056901592608768. Current Snapshot: MvccSnapshot[applied={T|T < 7113056901677805568}]
I20250111 09:02:37.642792 21418 tablet_replica.cc:440] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "741496baa99c4e189c56a345852c2ea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } }
I20250111 09:02:37.642868 21418 tablet_replica.cc:443] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:02:37.643951 21408 tablet_replica.cc:440] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "741496baa99c4e189c56a345852c2ea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } }
I20250111 09:02:37.644030 21408 tablet_replica.cc:443] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:37.644140 21435 mvcc.cc:204] Tried to move back new op lower bound from 7113056901677805568 to 7113056901592608768. Current Snapshot: MvccSnapshot[applied={T|T < 7113056901677805568}]
I20250111 09:02:37.644726 21407 tablet_replica.cc:440] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "741496baa99c4e189c56a345852c2ea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } }
I20250111 09:02:37.644804 21407 tablet_replica.cc:443] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:02:37.645870 21412 tablet_replica.cc:440] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "741496baa99c4e189c56a345852c2ea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } }
I20250111 09:02:37.645957 21412 tablet_replica.cc:443] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:37.667920 21054 raft_consensus.cc:1270] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Refusing update from remote peer 741496baa99c4e189c56a345852c2ea2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:02:37.667929 21337 raft_consensus.cc:1270] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11 [term 1 FOLLOWER]: Refusing update from remote peer 741496baa99c4e189c56a345852c2ea2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:02:37.668182 21418 consensus_queue.cc:1035] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:02:37.668287 21441 consensus_queue.cc:1035] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:02:37.675176 21436 mvcc.cc:204] Tried to move back new op lower bound from 7113056901805314048 to 7113056901047738368. Current Snapshot: MvccSnapshot[applied={T|T < 7113056901805314048}]
W20250111 09:02:37.720118 21451 tablet_replica.cc:1307] Illegal state: Failed to write to server: 741496baa99c4e189c56a345852c2ea2 (127.14.61.2:44379): transaction ID 0 not open: COMMITTED: cancelling pending write operations
W20250111 09:02:37.720245 21451 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45588: Illegal state: Failed to write to server: 741496baa99c4e189c56a345852c2ea2 (127.14.61.2:44379): transaction ID 0 not open: COMMITTED
W20250111 09:02:37.720813 21393 batcher.cc:441] Illegal state: Failed to write batch of 1 ops to tablet 1858ad7ec2ca435eb404436d8159656a after 1 attempt(s): Failed to write to server: 921d1c0189ff47bfa769e489f9d4ef11 (127.14.61.3:44997): Failed to write to server: 741496baa99c4e189c56a345852c2ea2 (127.14.61.2:44379): transaction ID 0 not open: COMMITTED
I20250111 09:02:37.753546 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 20979
I20250111 09:02:37.760187 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.1:36483
--local_ip_for_outbound_sockets=127.14.61.1
--tserver_master_addrs=127.14.61.62:43479
--webserver_port=41241
--webserver_interface=127.14.61.1
--builtin_ntp_servers=127.14.61.20:44415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_txn_system_client_init=true with env {}
W20250111 09:02:37.761740 21132 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.61.1:36483: connect: Connection refused (error 111)
W20250111 09:02:37.762589 21132 consensus_peers.cc:487] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 -> Peer f7ac20146503466cad420c7364808411 (127.14.61.1:36483): Couldn't send request to peer f7ac20146503466cad420c7364808411. Status: Network error: Client connection negotiation failed: client connection to 127.14.61.1:36483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250111 09:02:37.809360 21408 consensus_queue.cc:1035] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250111 09:02:37.813182 21269 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.61.1:36483: connect: Connection refused (error 111)
W20250111 09:02:37.813979 21269 consensus_peers.cc:487] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 -> Peer f7ac20146503466cad420c7364808411 (127.14.61.1:36483): Couldn't send request to peer f7ac20146503466cad420c7364808411. Status: Network error: Client connection negotiation failed: client connection to 127.14.61.1:36483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250111 09:02:37.858563 21454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:02:37.858723 21454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:02:37.858776 21454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:02:37.861750 21454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:02:37.861820 21454 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:02:37.861892 21454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.1
I20250111 09:02:37.865259 21454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:44415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.1:36483
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.61.1
--webserver_port=41241
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:43479
--never_fsync=true
--heap_profile_path=/tmp/kudu.21454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.1
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:02:37.866113 21454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:02:37.867082 21454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:37.874876 21465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:37.875483 21467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:37.876140 21464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:37.876855 21454 server_base.cc:1034] running on GCE node
I20250111 09:02:37.877193 21454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:02:37.877650 21454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:02:37.878779 21454 hybrid_clock.cc:648] HybridClock initialized: now 1736586157878767 us; error 27 us; skew 500 ppm
I20250111 09:02:37.880375 21454 webserver.cc:458] Webserver started at http://127.14.61.1:41241/ using document root <none> and password file <none>
I20250111 09:02:37.880867 21454 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:37.880935 21454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:37.883845 21454 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:37.885704 21473 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:37.886588 21454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20250111 09:02:37.886680 21454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/wal
uuid: "f7ac20146503466cad420c7364808411"
format_stamp: "Formatted at 2025-01-11 09:02:36 on dist-test-slave-nvbt"
I20250111 09:02:37.887022 21454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:37.895205 21454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:37.895598 21454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:02:37.895711 21454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:37.897212 21481 ts_tablet_manager.cc:536] Loading tablet metadata (0/3 complete)
I20250111 09:02:37.902333 21454 ts_tablet_manager.cc:579] Loaded tablet metadata (3 total tablets, 3 live tablets)
I20250111 09:02:37.902396 21454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20250111 09:02:37.902448 21454 ts_tablet_manager.cc:594] Registering tablets (0/3 complete)
I20250111 09:02:37.904606 21481 tablet_bootstrap.cc:492] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411: Bootstrap starting.
I20250111 09:02:37.905464 21454 ts_tablet_manager.cc:610] Registered 3 tablets
I20250111 09:02:37.905519 21454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.003s
I20250111 09:02:37.915045 21454 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:36483
I20250111 09:02:37.915200 21594 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:36483 every 8 connection(s)
I20250111 09:02:37.916008 21454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-0/data/info.pb
I20250111 09:02:37.917608 21595 heartbeater.cc:346] Connected to a master server at 127.14.61.62:43479
I20250111 09:02:37.917732 21595 heartbeater.cc:463] Registering TS with master...
I20250111 09:02:37.918212 21595 heartbeater.cc:510] Master 127.14.61.62:43479 requested a full tablet report, sending...
I20250111 09:02:37.919927 21481 log.cc:826] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411: Log is configured to *not* fsync() on all Append() calls
I20250111 09:02:37.920037 20907 ts_manager.cc:194] Re-registered known tserver with Master: f7ac20146503466cad420c7364808411 (127.14.61.1:36483)
I20250111 09:02:37.921185 20907 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.1:50983
I20250111 09:02:37.925345 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 21454
I20250111 09:02:37.925501 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 21116
I20250111 09:02:37.929318 21481 tablet_bootstrap.cc:492] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=10 ignored=0} inserts{seen=4 ignored=0} mutations{seen=5 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20250111 09:02:37.929642 21481 tablet_bootstrap.cc:492] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411: Bootstrap complete.
I20250111 09:02:37.930667 21481 ts_tablet_manager.cc:1397] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411: Time spent bootstrapping tablet: real 0.026s user 0.016s sys 0.007s
I20250111 09:02:37.932936 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.2:44379
--local_ip_for_outbound_sockets=127.14.61.2
--tserver_master_addrs=127.14.61.62:43479
--webserver_port=44933
--webserver_interface=127.14.61.2
--builtin_ntp_servers=127.14.61.20:44415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_txn_system_client_init=true with env {}
I20250111 09:02:37.933005 21481 raft_consensus.cc:357] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.934206 21481 raft_consensus.cc:738] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ac20146503466cad420c7364808411, State: Initialized, Role: FOLLOWER
I20250111 09:02:37.934617 21481 consensus_queue.cc:260] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.935075 21600 tablet_replica.cc:440] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } }
I20250111 09:02:37.935148 21595 heartbeater.cc:502] Master 127.14.61.62:43479 was elected leader, sending a full tablet report...
I20250111 09:02:37.935181 21600 tablet_replica.cc:443] This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:37.935356 21481 ts_tablet_manager.cc:1428] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20250111 09:02:37.935525 21481 tablet_bootstrap.cc:492] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411: Bootstrap starting.
I20250111 09:02:37.949935 21481 tablet_bootstrap.cc:492] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=2 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250111 09:02:37.950203 21481 tablet_bootstrap.cc:492] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411: Bootstrap complete.
I20250111 09:02:37.951068 21481 ts_tablet_manager.cc:1397] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.003s
I20250111 09:02:37.951409 21481 raft_consensus.cc:357] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } }
I20250111 09:02:37.951493 21481 raft_consensus.cc:738] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ac20146503466cad420c7364808411, State: Initialized, Role: FOLLOWER
I20250111 09:02:37.951620 21481 consensus_queue.cc:260] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } }
I20250111 09:02:37.951853 21481 ts_tablet_manager.cc:1428] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250111 09:02:37.952008 21481 tablet_bootstrap.cc:492] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411: Bootstrap starting.
I20250111 09:02:37.960201 21481 tablet_bootstrap.cc:492] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250111 09:02:37.960408 21481 tablet_bootstrap.cc:492] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411: Bootstrap complete.
I20250111 09:02:37.961292 21481 ts_tablet_manager.cc:1397] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.009s
I20250111 09:02:37.961606 21481 raft_consensus.cc:357] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.961684 21481 raft_consensus.cc:738] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ac20146503466cad420c7364808411, State: Initialized, Role: FOLLOWER
I20250111 09:02:37.961796 21481 consensus_queue.cc:260] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:37.962028 21481 ts_tablet_manager.cc:1428] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20250111 09:02:38.030495 21598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:02:38.030668 21598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:02:38.030710 21598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:02:38.033387 21598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:02:38.033444 21598 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:02:38.033483 21598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.2
I20250111 09:02:38.036598 21598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:44415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.2:44379
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.61.2
--webserver_port=44933
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:43479
--never_fsync=true
--heap_profile_path=/tmp/kudu.21598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.2
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:02:38.037333 21598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:02:38.038323 21598 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:38.045625 21610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:38.046186 21608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:38.046797 21607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:38.047492 21598 server_base.cc:1034] running on GCE node
I20250111 09:02:38.047746 21598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:02:38.048190 21598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:02:38.049319 21598 hybrid_clock.cc:648] HybridClock initialized: now 1736586158049309 us; error 27 us; skew 500 ppm
I20250111 09:02:38.050889 21598 webserver.cc:458] Webserver started at http://127.14.61.2:44933/ using document root <none> and password file <none>
I20250111 09:02:38.051355 21598 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:38.051422 21598 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:38.054018 21598 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20250111 09:02:38.055612 21616 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:38.056432 21598 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:38.056514 21598 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/wal
uuid: "741496baa99c4e189c56a345852c2ea2"
format_stamp: "Formatted at 2025-01-11 09:02:37 on dist-test-slave-nvbt"
I20250111 09:02:38.056831 21598 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:38.071965 21598 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:38.072338 21598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:02:38.072458 21598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:38.073805 21624 ts_tablet_manager.cc:536] Loading tablet metadata (0/3 complete)
I20250111 09:02:38.078401 21598 ts_tablet_manager.cc:579] Loaded tablet metadata (3 total tablets, 3 live tablets)
I20250111 09:02:38.078459 21598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20250111 09:02:38.078505 21598 ts_tablet_manager.cc:594] Registering tablets (0/3 complete)
I20250111 09:02:38.080340 21624 tablet_bootstrap.cc:492] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2: Bootstrap starting.
I20250111 09:02:38.081262 21598 ts_tablet_manager.cc:610] Registered 3 tablets
I20250111 09:02:38.081310 21598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20250111 09:02:38.091908 21598 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.2:44379
I20250111 09:02:38.091924 21737 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.2:44379 every 8 connection(s)
I20250111 09:02:38.093112 21598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-1/data/info.pb
I20250111 09:02:38.094255 21738 heartbeater.cc:346] Connected to a master server at 127.14.61.62:43479
I20250111 09:02:38.094355 21738 heartbeater.cc:463] Registering TS with master...
I20250111 09:02:38.094691 21738 heartbeater.cc:510] Master 127.14.61.62:43479 requested a full tablet report, sending...
I20250111 09:02:38.096014 20907 ts_manager.cc:194] Re-registered known tserver with Master: 741496baa99c4e189c56a345852c2ea2 (127.14.61.2:44379)
I20250111 09:02:38.096155 21624 log.cc:826] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2: Log is configured to *not* fsync() on all Append() calls
I20250111 09:02:38.096951 20907 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.2:39267
I20250111 09:02:38.098235 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 21598
I20250111 09:02:38.098405 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 21253
I20250111 09:02:38.105093 14580 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskFKQovg/build/debug/bin/kudu
/tmp/dist-test-taskFKQovg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.61.3:44997
--local_ip_for_outbound_sockets=127.14.61.3
--tserver_master_addrs=127.14.61.62:43479
--webserver_port=35389
--webserver_interface=127.14.61.3
--builtin_ntp_servers=127.14.61.20:44415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_txn_system_client_init=true with env {}
I20250111 09:02:38.106580 21624 tablet_bootstrap.cc:492] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=4 ignored=0} mutations{seen=6 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250111 09:02:38.106856 21624 tablet_bootstrap.cc:492] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2: Bootstrap complete.
I20250111 09:02:38.107753 21624 ts_tablet_manager.cc:1397] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2: Time spent bootstrapping tablet: real 0.028s user 0.015s sys 0.008s
I20250111 09:02:38.109753 21624 raft_consensus.cc:357] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:38.110038 21624 raft_consensus.cc:738] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 741496baa99c4e189c56a345852c2ea2, State: Initialized, Role: FOLLOWER
I20250111 09:02:38.110410 21624 consensus_queue.cc:260] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:38.110977 21738 heartbeater.cc:502] Master 127.14.61.62:43479 was elected leader, sending a full tablet report...
I20250111 09:02:38.110926 21742 tablet_replica.cc:440] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } }
I20250111 09:02:38.111097 21624 ts_tablet_manager.cc:1428] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2: Time spent starting tablet: real 0.003s user 0.006s sys 0.000s
I20250111 09:02:38.111187 21742 tablet_replica.cc:443] This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:38.111377 21624 tablet_bootstrap.cc:492] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2: Bootstrap starting.
I20250111 09:02:38.124462 21624 tablet_bootstrap.cc:492] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=2 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250111 09:02:38.124688 21624 tablet_bootstrap.cc:492] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2: Bootstrap complete.
I20250111 09:02:38.125519 21624 ts_tablet_manager.cc:1397] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.006s
I20250111 09:02:38.125833 21624 raft_consensus.cc:357] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } }
I20250111 09:02:38.125914 21624 raft_consensus.cc:738] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 741496baa99c4e189c56a345852c2ea2, State: Initialized, Role: FOLLOWER
I20250111 09:02:38.126032 21624 consensus_queue.cc:260] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } }
I20250111 09:02:38.126271 21624 ts_tablet_manager.cc:1428] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250111 09:02:38.126433 21624 tablet_bootstrap.cc:492] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2: Bootstrap starting.
I20250111 09:02:38.134744 21624 tablet_bootstrap.cc:492] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250111 09:02:38.134995 21624 tablet_bootstrap.cc:492] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2: Bootstrap complete.
I20250111 09:02:38.135805 21624 ts_tablet_manager.cc:1397] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20250111 09:02:38.136121 21624 raft_consensus.cc:357] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:38.136206 21624 raft_consensus.cc:738] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 741496baa99c4e189c56a345852c2ea2, State: Initialized, Role: FOLLOWER
I20250111 09:02:38.136338 21624 consensus_queue.cc:260] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:38.136541 21624 ts_tablet_manager.cc:1428] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
W20250111 09:02:38.167053 21596 tablet.cc:2367] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250111 09:02:38.200304 21741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 09:02:38.200474 21741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 09:02:38.200543 21741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 09:02:38.203385 21741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 09:02:38.203464 21741 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20250111 09:02:38.203538 21741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.61.3
I20250111 09:02:38.206876 21741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.61.20:44415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.61.3:44997
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.61.3
--webserver_port=35389
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.61.62:43479
--never_fsync=true
--heap_profile_path=/tmp/kudu.21741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.61.3
--log_dir=/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type DEBUG
built by None at 11 Jan 2025 08:43:13 UTC on 24a791456cd2
build id 4121
I20250111 09:02:38.207675 21741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:02:38.208600 21741 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:38.215603 21752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:38.216243 21749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:38.216863 21750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:38.217550 21741 server_base.cc:1034] running on GCE node
I20250111 09:02:38.217783 21741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 09:02:38.218139 21741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 09:02:38.219267 21741 hybrid_clock.cc:648] HybridClock initialized: now 1736586158219249 us; error 30 us; skew 500 ppm
I20250111 09:02:38.220791 21741 webserver.cc:458] Webserver started at http://127.14.61.3:35389/ using document root <none> and password file <none>
I20250111 09:02:38.221284 21741 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:38.221341 21741 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:38.223994 21741 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20250111 09:02:38.225725 21758 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:38.226599 21741 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20250111 09:02:38.226678 21741 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/wal
uuid: "921d1c0189ff47bfa769e489f9d4ef11"
format_stamp: "Formatted at 2025-01-11 09:02:37 on dist-test-slave-nvbt"
I20250111 09:02:38.227015 21741 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:38.241050 21741 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:38.241485 21741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:02:38.241592 21741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:38.243057 21766 ts_tablet_manager.cc:536] Loading tablet metadata (0/3 complete)
I20250111 09:02:38.248071 21741 ts_tablet_manager.cc:579] Loaded tablet metadata (3 total tablets, 3 live tablets)
I20250111 09:02:38.248119 21741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20250111 09:02:38.248178 21741 ts_tablet_manager.cc:594] Registering tablets (0/3 complete)
I20250111 09:02:38.250075 21766 tablet_bootstrap.cc:492] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11: Bootstrap starting.
I20250111 09:02:38.250950 21741 ts_tablet_manager.cc:610] Registered 3 tablets
I20250111 09:02:38.251003 21741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20250111 09:02:38.260197 21741 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.3:44997
I20250111 09:02:38.260227 21879 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.3:44997 every 8 connection(s)
I20250111 09:02:38.261487 21741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnWriteOpsITest.TxnWriteAfterCommit.1736586070622223-14580-0/minicluster-data/ts-2/data/info.pb
I20250111 09:02:38.262627 21880 heartbeater.cc:346] Connected to a master server at 127.14.61.62:43479
I20250111 09:02:38.262732 21880 heartbeater.cc:463] Registering TS with master...
I20250111 09:02:38.263064 21880 heartbeater.cc:510] Master 127.14.61.62:43479 requested a full tablet report, sending...
I20250111 09:02:38.264408 20907 ts_manager.cc:194] Re-registered known tserver with Master: 921d1c0189ff47bfa769e489f9d4ef11 (127.14.61.3:44997)
I20250111 09:02:38.265377 20907 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.61.3:40249
I20250111 09:02:38.265661 21766 log.cc:826] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11: Log is configured to *not* fsync() on all Append() calls
I20250111 09:02:38.270308 14580 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskFKQovg/build/debug/bin/kudu as pid 21741
I20250111 09:02:38.273768 21766 tablet_bootstrap.cc:492] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=4 ignored=0} mutations{seen=6 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250111 09:02:38.274083 21766 tablet_bootstrap.cc:492] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11: Bootstrap complete.
I20250111 09:02:38.275203 21766 ts_tablet_manager.cc:1397] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11: Time spent bootstrapping tablet: real 0.025s user 0.012s sys 0.011s
I20250111 09:02:38.277316 21766 raft_consensus.cc:357] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:38.277585 21766 raft_consensus.cc:738] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921d1c0189ff47bfa769e489f9d4ef11, State: Initialized, Role: FOLLOWER
I20250111 09:02:38.277981 21766 consensus_queue.cc:260] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:38.278445 21886 tablet_replica.cc:440] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } }
I20250111 09:02:38.278493 21880 heartbeater.cc:502] Master 127.14.61.62:43479 was elected leader, sending a full tablet report...
I20250111 09:02:38.278544 21886 tablet_replica.cc:443] This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:38.278718 21766 ts_tablet_manager.cc:1428] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11: Time spent starting tablet: real 0.003s user 0.003s sys 0.002s
I20250111 09:02:38.278960 21766 tablet_bootstrap.cc:492] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11: Bootstrap starting.
I20250111 09:02:38.292922 21766 tablet_bootstrap.cc:492] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=2 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250111 09:02:38.293234 21766 tablet_bootstrap.cc:492] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11: Bootstrap complete.
I20250111 09:02:38.294173 21766 ts_tablet_manager.cc:1397] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.011s
I20250111 09:02:38.294591 21766 raft_consensus.cc:357] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } }
I20250111 09:02:38.294690 21766 raft_consensus.cc:738] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921d1c0189ff47bfa769e489f9d4ef11, State: Initialized, Role: FOLLOWER
I20250111 09:02:38.294808 21766 consensus_queue.cc:260] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } }
I20250111 09:02:38.295035 21766 ts_tablet_manager.cc:1428] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250111 09:02:38.295174 21766 tablet_bootstrap.cc:492] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11: Bootstrap starting.
I20250111 09:02:38.303143 21766 tablet_bootstrap.cc:492] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250111 09:02:38.303406 21766 tablet_bootstrap.cc:492] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11: Bootstrap complete.
I20250111 09:02:38.304314 21766 ts_tablet_manager.cc:1397] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20250111 09:02:38.304602 21766 raft_consensus.cc:357] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:38.304692 21766 raft_consensus.cc:738] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921d1c0189ff47bfa769e489f9d4ef11, State: Initialized, Role: FOLLOWER
I20250111 09:02:38.304790 21766 consensus_queue.cc:260] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:38.304972 21766 ts_tablet_manager.cc:1428] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250111 09:02:39.112802 21892 raft_consensus.cc:491] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:02:39.112942 21892 raft_consensus.cc:513] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:39.113799 21892 leader_election.cc:290] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 741496baa99c4e189c56a345852c2ea2 (127.14.61.2:44379), 921d1c0189ff47bfa769e489f9d4ef11 (127.14.61.3:44997)
I20250111 09:02:39.117657 21692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ccc398fb2f4029b2c7acc60252dfab" candidate_uuid: "f7ac20146503466cad420c7364808411" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: false dest_uuid: "741496baa99c4e189c56a345852c2ea2" is_pre_election: true
I20250111 09:02:39.117697 21825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ccc398fb2f4029b2c7acc60252dfab" candidate_uuid: "f7ac20146503466cad420c7364808411" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: false dest_uuid: "921d1c0189ff47bfa769e489f9d4ef11" is_pre_election: true
I20250111 09:02:39.117883 21692 raft_consensus.cc:2463] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7ac20146503466cad420c7364808411 in term 1.
I20250111 09:02:39.117941 21825 raft_consensus.cc:2463] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7ac20146503466cad420c7364808411 in term 1.
I20250111 09:02:39.118229 21477 leader_election.cc:304] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 741496baa99c4e189c56a345852c2ea2, f7ac20146503466cad420c7364808411; no voters:
I20250111 09:02:39.118428 21892 raft_consensus.cc:2798] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250111 09:02:39.118495 21892 raft_consensus.cc:491] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:02:39.118551 21892 raft_consensus.cc:3054] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Advancing to term 2
I20250111 09:02:39.120085 21892 raft_consensus.cc:513] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:39.120369 21892 leader_election.cc:290] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [CANDIDATE]: Term 2 election: Requested vote from peers 741496baa99c4e189c56a345852c2ea2 (127.14.61.2:44379), 921d1c0189ff47bfa769e489f9d4ef11 (127.14.61.3:44997)
I20250111 09:02:39.120673 21692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ccc398fb2f4029b2c7acc60252dfab" candidate_uuid: "f7ac20146503466cad420c7364808411" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: false dest_uuid: "741496baa99c4e189c56a345852c2ea2"
I20250111 09:02:39.120735 21825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ccc398fb2f4029b2c7acc60252dfab" candidate_uuid: "f7ac20146503466cad420c7364808411" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: false dest_uuid: "921d1c0189ff47bfa769e489f9d4ef11"
I20250111 09:02:39.120798 21692 raft_consensus.cc:3054] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Advancing to term 2
I20250111 09:02:39.120826 21825 raft_consensus.cc:3054] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11 [term 1 FOLLOWER]: Advancing to term 2
I20250111 09:02:39.122160 21692 raft_consensus.cc:2463] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7ac20146503466cad420c7364808411 in term 2.
I20250111 09:02:39.122320 21825 raft_consensus.cc:2463] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7ac20146503466cad420c7364808411 in term 2.
I20250111 09:02:39.122427 21477 leader_election.cc:304] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 741496baa99c4e189c56a345852c2ea2, f7ac20146503466cad420c7364808411; no voters:
I20250111 09:02:39.122576 21892 raft_consensus.cc:2798] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [term 2 FOLLOWER]: Leader election won for term 2
I20250111 09:02:39.122743 21892 raft_consensus.cc:695] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [term 2 LEADER]: Becoming Leader. State: Replica: f7ac20146503466cad420c7364808411, State: Running, Role: LEADER
I20250111 09:02:39.123018 21892 consensus_queue.cc:237] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.11, Last appended by leader: 11, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:39.124815 21897 tablet_replica.cc:440] TxnStatusTablet state changed. Reason: New leader f7ac20146503466cad420c7364808411. Latest consensus state: current_term: 2 leader_uuid: "f7ac20146503466cad420c7364808411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } }
I20250111 09:02:39.124907 21897 tablet_replica.cc:443] This TxnStatusTablet replica's current role is: LEADER
I20250111 09:02:39.125136 21899 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:02:39.125873 20907 catalog_manager.cc:5526] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 reported cstate change: term changed from 1 to 2, leader changed from 741496baa99c4e189c56a345852c2ea2 (127.14.61.2) to f7ac20146503466cad420c7364808411 (127.14.61.1). New cstate: current_term: 2 leader_uuid: "f7ac20146503466cad420c7364808411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:39.373543 21904 raft_consensus.cc:491] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:02:39.373646 21904 raft_consensus.cc:513] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } }
I20250111 09:02:39.374547 21904 leader_election.cc:290] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921d1c0189ff47bfa769e489f9d4ef11 (127.14.61.3:44997), f7ac20146503466cad420c7364808411 (127.14.61.1:36483)
I20250111 09:02:39.378437 21825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "470943c2f9784db0a50bb8d4c207c1a3" candidate_uuid: "741496baa99c4e189c56a345852c2ea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "921d1c0189ff47bfa769e489f9d4ef11" is_pre_election: true
I20250111 09:02:39.378587 21825 raft_consensus.cc:2463] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 741496baa99c4e189c56a345852c2ea2 in term 1.
I20250111 09:02:39.378661 21549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "470943c2f9784db0a50bb8d4c207c1a3" candidate_uuid: "741496baa99c4e189c56a345852c2ea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "f7ac20146503466cad420c7364808411" is_pre_election: true
I20250111 09:02:39.378871 21549 raft_consensus.cc:2463] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 741496baa99c4e189c56a345852c2ea2 in term 1.
I20250111 09:02:39.378906 21620 leader_election.cc:304] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 741496baa99c4e189c56a345852c2ea2, 921d1c0189ff47bfa769e489f9d4ef11; no voters:
I20250111 09:02:39.379158 21904 raft_consensus.cc:2798] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250111 09:02:39.379230 21904 raft_consensus.cc:491] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:02:39.379289 21904 raft_consensus.cc:3054] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Advancing to term 2
I20250111 09:02:39.380079 21904 raft_consensus.cc:513] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } }
I20250111 09:02:39.380386 21904 leader_election.cc:290] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [CANDIDATE]: Term 2 election: Requested vote from peers 921d1c0189ff47bfa769e489f9d4ef11 (127.14.61.3:44997), f7ac20146503466cad420c7364808411 (127.14.61.1:36483)
I20250111 09:02:39.380677 21825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "470943c2f9784db0a50bb8d4c207c1a3" candidate_uuid: "741496baa99c4e189c56a345852c2ea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "921d1c0189ff47bfa769e489f9d4ef11"
I20250111 09:02:39.380723 21549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "470943c2f9784db0a50bb8d4c207c1a3" candidate_uuid: "741496baa99c4e189c56a345852c2ea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "f7ac20146503466cad420c7364808411"
I20250111 09:02:39.380785 21825 raft_consensus.cc:3054] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11 [term 1 FOLLOWER]: Advancing to term 2
I20250111 09:02:39.380816 21549 raft_consensus.cc:3054] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Advancing to term 2
I20250111 09:02:39.381520 21825 raft_consensus.cc:2463] T 470943c2f9784db0a50bb8d4c207c1a3 P 921d1c0189ff47bfa769e489f9d4ef11 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 741496baa99c4e189c56a345852c2ea2 in term 2.
I20250111 09:02:39.381577 21549 raft_consensus.cc:2463] T 470943c2f9784db0a50bb8d4c207c1a3 P f7ac20146503466cad420c7364808411 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 741496baa99c4e189c56a345852c2ea2 in term 2.
I20250111 09:02:39.381763 21620 leader_election.cc:304] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 741496baa99c4e189c56a345852c2ea2, 921d1c0189ff47bfa769e489f9d4ef11; no voters:
I20250111 09:02:39.381904 21904 raft_consensus.cc:2798] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [term 2 FOLLOWER]: Leader election won for term 2
I20250111 09:02:39.382122 21904 raft_consensus.cc:695] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [term 2 LEADER]: Becoming Leader. State: Replica: 741496baa99c4e189c56a345852c2ea2, State: Running, Role: LEADER
I20250111 09:02:39.382370 21904 consensus_queue.cc:237] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } }
I20250111 09:02:39.384819 20907 catalog_manager.cc:5526] T 470943c2f9784db0a50bb8d4c207c1a3 P 741496baa99c4e189c56a345852c2ea2 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "741496baa99c4e189c56a345852c2ea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } health_report { overall_health: UNKNOWN } } }
I20250111 09:02:39.605075 21825 raft_consensus.cc:1270] T 20ccc398fb2f4029b2c7acc60252dfab P 921d1c0189ff47bfa769e489f9d4ef11 [term 2 FOLLOWER]: Refusing update from remote peer f7ac20146503466cad420c7364808411: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 2 index: 12. (index mismatch)
I20250111 09:02:39.605713 21897 consensus_queue.cc:1035] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20250111 09:02:39.607503 21910 tablet_replica.cc:440] TxnStatusTablet state changed. Reason: New leader f7ac20146503466cad420c7364808411. Latest consensus state: current_term: 2 leader_uuid: "f7ac20146503466cad420c7364808411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } }
I20250111 09:02:39.607599 21910 tablet_replica.cc:443] This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:39.609062 21892 tablet_replica.cc:440] TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "f7ac20146503466cad420c7364808411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } }
I20250111 09:02:39.609164 21892 tablet_replica.cc:443] This TxnStatusTablet replica's current role is: LEADER
I20250111 09:02:39.609496 21910 tablet_replica.cc:440] TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "f7ac20146503466cad420c7364808411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } }
I20250111 09:02:39.609584 21910 tablet_replica.cc:443] This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:39.610231 21692 raft_consensus.cc:1270] T 20ccc398fb2f4029b2c7acc60252dfab P 741496baa99c4e189c56a345852c2ea2 [term 2 FOLLOWER]: Refusing update from remote peer f7ac20146503466cad420c7364808411: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 2 index: 12. (index mismatch)
I20250111 09:02:39.610682 21892 consensus_queue.cc:1035] T 20ccc398fb2f4029b2c7acc60252dfab P f7ac20146503466cad420c7364808411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20250111 09:02:39.611979 21908 tablet_replica.cc:440] TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "f7ac20146503466cad420c7364808411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } }
I20250111 09:02:39.612085 21908 tablet_replica.cc:443] This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:39.612082 21904 tablet_replica.cc:440] TxnStatusTablet state changed. Reason: New leader f7ac20146503466cad420c7364808411. Latest consensus state: current_term: 2 leader_uuid: "f7ac20146503466cad420c7364808411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } } }
I20250111 09:02:39.612179 21904 tablet_replica.cc:443] This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:02:39.640654 21899 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:02:39.651417 21904 raft_consensus.cc:491] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:02:39.651536 21904 raft_consensus.cc:513] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:39.651580 21892 raft_consensus.cc:491] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:02:39.651659 21892 raft_consensus.cc:513] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:39.651914 21904 leader_election.cc:290] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f7ac20146503466cad420c7364808411 (127.14.61.1:36483), 921d1c0189ff47bfa769e489f9d4ef11 (127.14.61.3:44997)
I20250111 09:02:39.651981 21892 leader_election.cc:290] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 741496baa99c4e189c56a345852c2ea2 (127.14.61.2:44379), 921d1c0189ff47bfa769e489f9d4ef11 (127.14.61.3:44997)
I20250111 09:02:39.652284 21825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1858ad7ec2ca435eb404436d8159656a" candidate_uuid: "741496baa99c4e189c56a345852c2ea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921d1c0189ff47bfa769e489f9d4ef11" is_pre_election: true
I20250111 09:02:39.652349 21692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1858ad7ec2ca435eb404436d8159656a" candidate_uuid: "f7ac20146503466cad420c7364808411" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "741496baa99c4e189c56a345852c2ea2" is_pre_election: true
I20250111 09:02:39.652415 21825 raft_consensus.cc:2463] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 741496baa99c4e189c56a345852c2ea2 in term 1.
I20250111 09:02:39.652467 21692 raft_consensus.cc:2405] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f7ac20146503466cad420c7364808411 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:02:39.652437 21549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1858ad7ec2ca435eb404436d8159656a" candidate_uuid: "741496baa99c4e189c56a345852c2ea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f7ac20146503466cad420c7364808411" is_pre_election: true
I20250111 09:02:39.652556 21549 raft_consensus.cc:2463] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 741496baa99c4e189c56a345852c2ea2 in term 1.
I20250111 09:02:39.652693 21620 leader_election.cc:304] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 741496baa99c4e189c56a345852c2ea2, 921d1c0189ff47bfa769e489f9d4ef11; no voters:
I20250111 09:02:39.652837 21904 raft_consensus.cc:2798] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250111 09:02:39.652884 21904 raft_consensus.cc:491] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:02:39.652920 21904 raft_consensus.cc:3054] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [term 1 FOLLOWER]: Advancing to term 2
I20250111 09:02:39.653069 21825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1858ad7ec2ca435eb404436d8159656a" candidate_uuid: "f7ac20146503466cad420c7364808411" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921d1c0189ff47bfa769e489f9d4ef11" is_pre_election: true
I20250111 09:02:39.653198 21825 raft_consensus.cc:2405] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f7ac20146503466cad420c7364808411 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250111 09:02:39.653425 21477 leader_election.cc:304] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f7ac20146503466cad420c7364808411; no voters: 741496baa99c4e189c56a345852c2ea2, 921d1c0189ff47bfa769e489f9d4ef11
I20250111 09:02:39.653589 21892 raft_consensus.cc:2743] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250111 09:02:39.653892 21904 raft_consensus.cc:513] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:39.654132 21904 leader_election.cc:290] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [CANDIDATE]: Term 2 election: Requested vote from peers f7ac20146503466cad420c7364808411 (127.14.61.1:36483), 921d1c0189ff47bfa769e489f9d4ef11 (127.14.61.3:44997)
I20250111 09:02:39.654389 21825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1858ad7ec2ca435eb404436d8159656a" candidate_uuid: "741496baa99c4e189c56a345852c2ea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921d1c0189ff47bfa769e489f9d4ef11"
I20250111 09:02:39.654417 21549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1858ad7ec2ca435eb404436d8159656a" candidate_uuid: "741496baa99c4e189c56a345852c2ea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f7ac20146503466cad420c7364808411"
I20250111 09:02:39.654484 21825 raft_consensus.cc:3054] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [term 1 FOLLOWER]: Advancing to term 2
I20250111 09:02:39.654508 21549 raft_consensus.cc:3054] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411 [term 1 FOLLOWER]: Advancing to term 2
I20250111 09:02:39.655335 21549 raft_consensus.cc:2463] T 1858ad7ec2ca435eb404436d8159656a P f7ac20146503466cad420c7364808411 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 741496baa99c4e189c56a345852c2ea2 in term 2.
I20250111 09:02:39.655335 21825 raft_consensus.cc:2463] T 1858ad7ec2ca435eb404436d8159656a P 921d1c0189ff47bfa769e489f9d4ef11 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 741496baa99c4e189c56a345852c2ea2 in term 2.
I20250111 09:02:39.655576 21618 leader_election.cc:304] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 741496baa99c4e189c56a345852c2ea2, f7ac20146503466cad420c7364808411; no voters:
I20250111 09:02:39.655810 21904 raft_consensus.cc:2798] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [term 2 FOLLOWER]: Leader election won for term 2
I20250111 09:02:39.655901 21904 raft_consensus.cc:695] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [term 2 LEADER]: Becoming Leader. State: Replica: 741496baa99c4e189c56a345852c2ea2, State: Running, Role: LEADER
I20250111 09:02:39.656040 21904 consensus_queue.cc:237] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } }
I20250111 09:02:39.657356 20909 catalog_manager.cc:5526] T 1858ad7ec2ca435eb404436d8159656a P 741496baa99c4e189c56a345852c2ea2 reported cstate change: term changed from 1 to 2, leader changed from 921d1c0189ff47bfa769e489f9d4ef11 (127.14.61.3) to 741496baa99c4e189c56a345852c2ea2 (127.14.61.2). New cstate: current_term: 2 leader_uuid: "741496baa99c4e189c56a345852c2ea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ac20146503466cad420c7364808411" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "741496baa99c4e189c56a345852c2ea2" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 44379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "921d1c0189ff47bfa769e489f9d4ef11" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 44997 } health_report { overall_health: UNKNOWN } } }
W20250111 09:02:39.781986 21885 tablet_replica.cc:1307] Illegal state: Failed to write to server: f7ac20146503466cad420c7364808411 (127.14.61.1:36483): transaction ID 1 not open: COMMITTED: cancelling pending write operations
W20250111 09:02:39.782123 21885 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43692: Illegal state: Failed to write to server: f7ac20146503466cad420c7364808411 (127.14.61.1:36483): transaction ID 1 not open: COMMITTED
I20250111 09:02:39.782370 21885 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43692 (ReqId={client: 013798c6410542e3bce752479d86c838, seq_no=3, attempt_no=0}) took 1507 ms. Trace:
I20250111 09:02:39.782421 21885 rpcz_store.cc:276] 0111 09:02:38.275086 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:38.275187 (+ 101us) service_pool.cc:225] Handling call
0111 09:02:39.782359 (+1507172us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":93,"threads_started":1,"txn-participant-registration.queue_time_us":189}
W20250111 09:02:39.783139 21393 batcher.cc:441] Illegal state: Failed to write batch of 1 ops to tablet 470943c2f9784db0a50bb8d4c207c1a3 after 1 attempt(s): Failed to write to server: 741496baa99c4e189c56a345852c2ea2 (127.14.61.2:44379): Failed to write to server: f7ac20146503466cad420c7364808411 (127.14.61.1:36483): transaction ID 1 not open: COMMITTED
I20250111 09:02:39.783530 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 21454
I20250111 09:02:39.790292 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 21598
I20250111 09:02:39.795868 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 21741
I20250111 09:02:39.800935 14580 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskFKQovg/build/debug/bin/kudu with pid 20878
2025-01-11T09:02:39Z chronyd exiting
[ OK ] TxnWriteOpsITest.TxnWriteAfterCommit (3291 ms)
[----------] 8 tests from TxnWriteOpsITest (89191 ms total)
[----------] 18 tests from TxnOpDispatcherITest
[ RUN ] TxnOpDispatcherITest.LifecycleBasic
I20250111 09:02:39.822741 14580 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.61.62:33405
I20250111 09:02:39.823540 14580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 09:02:39.823943 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:39.828816 21921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:39.829483 21920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:39.830156 21923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:39.830857 14580 server_base.cc:1034] running on GCE node
I20250111 09:02:39.831285 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:02:39.831372 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:02:39.831403 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586159831400 us; error 0 us; skew 500 ppm
I20250111 09:02:39.832705 14580 webserver.cc:458] Webserver started at http://127.14.61.62:44041/ using document root <none> and password file <none>
I20250111 09:02:39.833257 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:39.833312 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:39.833541 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:02:39.834542 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.LifecycleBasic.1736586070622223-14580-0/minicluster-data/master-0-root/instance:
uuid: "0a42f82d46734a2a8e36424c339e42d1"
format_stamp: "Formatted at 2025-01-11 09:02:39 on dist-test-slave-nvbt"
I20250111 09:02:39.837431 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20250111 09:02:39.839424 21928 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:39.840256 14580 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:39.840358 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.LifecycleBasic.1736586070622223-14580-0/minicluster-data/master-0-root
uuid: "0a42f82d46734a2a8e36424c339e42d1"
format_stamp: "Formatted at 2025-01-11 09:02:39 on dist-test-slave-nvbt"
I20250111 09:02:39.840440 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.LifecycleBasic.1736586070622223-14580-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.LifecycleBasic.1736586070622223-14580-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.LifecycleBasic.1736586070622223-14580-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:39.855732 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:39.856102 14580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 09:02:39.856242 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:39.862238 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:33405
I20250111 09:02:39.862603 21990 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:33405 every 8 connection(s)
I20250111 09:02:39.863852 21991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:39.868306 21991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1: Bootstrap starting.
I20250111 09:02:39.869889 21991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:39.870649 21991 log.cc:826] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1: Log is configured to *not* fsync() on all Append() calls
I20250111 09:02:39.871909 21991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1: No bootstrap required, opened a new log
I20250111 09:02:39.873917 21991 raft_consensus.cc:357] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a42f82d46734a2a8e36424c339e42d1" member_type: VOTER }
I20250111 09:02:39.874043 21991 raft_consensus.cc:383] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:39.874114 21991 raft_consensus.cc:738] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a42f82d46734a2a8e36424c339e42d1, State: Initialized, Role: FOLLOWER
I20250111 09:02:39.874472 21991 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a42f82d46734a2a8e36424c339e42d1" member_type: VOTER }
I20250111 09:02:39.874585 21991 raft_consensus.cc:397] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:02:39.874641 21991 raft_consensus.cc:491] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:02:39.874743 21991 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:39.875376 21991 raft_consensus.cc:513] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a42f82d46734a2a8e36424c339e42d1" member_type: VOTER }
I20250111 09:02:39.875689 21991 leader_election.cc:304] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a42f82d46734a2a8e36424c339e42d1; no voters:
I20250111 09:02:39.875909 21991 leader_election.cc:290] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:02:39.876003 21994 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:39.876175 21994 raft_consensus.cc:695] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1 [term 1 LEADER]: Becoming Leader. State: Replica: 0a42f82d46734a2a8e36424c339e42d1, State: Running, Role: LEADER
I20250111 09:02:39.876459 21994 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a42f82d46734a2a8e36424c339e42d1" member_type: VOTER }
I20250111 09:02:39.876669 21991 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1 [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:02:39.877923 21996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a42f82d46734a2a8e36424c339e42d1. Latest consensus state: current_term: 1 leader_uuid: "0a42f82d46734a2a8e36424c339e42d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a42f82d46734a2a8e36424c339e42d1" member_type: VOTER } }
I20250111 09:02:39.877892 21995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a42f82d46734a2a8e36424c339e42d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a42f82d46734a2a8e36424c339e42d1" member_type: VOTER } }
I20250111 09:02:39.878011 21996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1 [sys.catalog]: This master's current role is: LEADER
I20250111 09:02:39.878034 21995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1 [sys.catalog]: This master's current role is: LEADER
I20250111 09:02:39.878504 22004 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:02:39.878582 14580 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250111 09:02:39.881160 22004 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:02:39.885457 22004 catalog_manager.cc:1348] Generated new cluster ID: 9bc063d056774ab3be55d468fc2fcb9d
I20250111 09:02:39.885522 22004 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:02:39.892554 22004 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:02:39.893189 22004 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:02:39.900000 22004 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1: Generated new TSK 0
I20250111 09:02:39.900595 22004 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:02:39.906510 21945 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57630:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20250111 09:02:39.910379 21991 master.cc:440] Invalid argument: unable to initialize TxnManager: Error creating table kudu_system.kudu_transactions on the master: not enough live tablet servers to create a table with the requested replication factor 1; 0 tablet servers are alive: unable to init TxnManager, will retry in 1.000s
I20250111 09:02:39.911011 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:39.913527 22022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:39.914090 22019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:39.914763 22020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:39.915331 14580 server_base.cc:1034] running on GCE node
I20250111 09:02:39.915477 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:02:39.915513 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:02:39.915527 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586159915527 us; error 0 us; skew 500 ppm
I20250111 09:02:39.916607 14580 webserver.cc:458] Webserver started at http://127.14.61.1:43519/ using document root <none> and password file <none>
I20250111 09:02:39.916743 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:39.916782 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:39.916844 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:02:39.917194 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.LifecycleBasic.1736586070622223-14580-0/minicluster-data/ts-0-root/instance:
uuid: "7f146b4127824a07a5ce867d82511cfe"
format_stamp: "Formatted at 2025-01-11 09:02:39 on dist-test-slave-nvbt"
I20250111 09:02:39.918388 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20250111 09:02:39.919139 22027 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:39.919369 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20250111 09:02:39.919421 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.LifecycleBasic.1736586070622223-14580-0/minicluster-data/ts-0-root
uuid: "7f146b4127824a07a5ce867d82511cfe"
format_stamp: "Formatted at 2025-01-11 09:02:39 on dist-test-slave-nvbt"
I20250111 09:02:39.919479 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.LifecycleBasic.1736586070622223-14580-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.LifecycleBasic.1736586070622223-14580-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.LifecycleBasic.1736586070622223-14580-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:39.933722 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:39.934002 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:39.935040 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:02:39.935101 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:39.935148 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:02:39.935177 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:39.941115 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:42633
I20250111 09:02:39.942044 22097 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:42633 every 8 connection(s)
I20250111 09:02:39.942368 22098 heartbeater.cc:346] Connected to a master server at 127.14.61.62:33405
I20250111 09:02:39.942480 22098 heartbeater.cc:463] Registering TS with master...
I20250111 09:02:39.942742 22098 heartbeater.cc:510] Master 127.14.61.62:33405 requested a full tablet report, sending...
I20250111 09:02:39.943895 21945 ts_manager.cc:194] Registered new tserver with Master: 7f146b4127824a07a5ce867d82511cfe (127.14.61.1:42633)
I20250111 09:02:39.943986 14580 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002265983s
I20250111 09:02:39.945277 21945 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57636
I20250111 09:02:39.950485 21945 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57656:
name: "txn_op_dispatcher_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
I20250111 09:02:39.960412 22049 tablet_service.cc:1467] Processing CreateTablet for tablet 11a48dd21ace4218a5f46fb20805d46d (DEFAULT_TABLE table=txn_op_dispatcher_test [id=327ba382bb3d48c48f7ba24c9aac7e81]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:39.960438 22048 tablet_service.cc:1467] Processing CreateTablet for tablet 90760889537647fd81890edf2ae2f5cb (DEFAULT_TABLE table=txn_op_dispatcher_test [id=327ba382bb3d48c48f7ba24c9aac7e81]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:39.960827 22049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11a48dd21ace4218a5f46fb20805d46d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:39.960983 22048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90760889537647fd81890edf2ae2f5cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:39.963266 22108 tablet_bootstrap.cc:492] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe: Bootstrap starting.
I20250111 09:02:39.964090 22108 tablet_bootstrap.cc:654] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:39.965281 22108 tablet_bootstrap.cc:492] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe: No bootstrap required, opened a new log
I20250111 09:02:39.965368 22108 ts_tablet_manager.cc:1397] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:39.965709 22108 raft_consensus.cc:357] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f146b4127824a07a5ce867d82511cfe" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42633 } }
I20250111 09:02:39.965792 22108 raft_consensus.cc:383] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:39.965828 22108 raft_consensus.cc:738] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f146b4127824a07a5ce867d82511cfe, State: Initialized, Role: FOLLOWER
I20250111 09:02:39.965938 22108 consensus_queue.cc:260] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f146b4127824a07a5ce867d82511cfe" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42633 } }
I20250111 09:02:39.966006 22108 raft_consensus.cc:397] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:02:39.966048 22108 raft_consensus.cc:491] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:02:39.966102 22108 raft_consensus.cc:3054] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:39.966723 22108 raft_consensus.cc:513] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f146b4127824a07a5ce867d82511cfe" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42633 } }
I20250111 09:02:39.966825 22108 leader_election.cc:304] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f146b4127824a07a5ce867d82511cfe; no voters:
I20250111 09:02:39.967000 22108 leader_election.cc:290] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:02:39.967139 22110 raft_consensus.cc:2798] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:39.967340 22110 raft_consensus.cc:695] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe [term 1 LEADER]: Becoming Leader. State: Replica: 7f146b4127824a07a5ce867d82511cfe, State: Running, Role: LEADER
I20250111 09:02:39.967371 22108 ts_tablet_manager.cc:1428] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250111 09:02:39.967466 22098 heartbeater.cc:502] Master 127.14.61.62:33405 was elected leader, sending a full tablet report...
I20250111 09:02:39.967660 22108 tablet_bootstrap.cc:492] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe: Bootstrap starting.
I20250111 09:02:39.967510 22110 consensus_queue.cc:237] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f146b4127824a07a5ce867d82511cfe" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42633 } }
I20250111 09:02:39.968554 22108 tablet_bootstrap.cc:654] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:39.969552 22108 tablet_bootstrap.cc:492] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe: No bootstrap required, opened a new log
I20250111 09:02:39.969623 22108 ts_tablet_manager.cc:1397] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250111 09:02:39.969960 22108 raft_consensus.cc:357] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f146b4127824a07a5ce867d82511cfe" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42633 } }
I20250111 09:02:39.970039 22108 raft_consensus.cc:383] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:39.970067 22108 raft_consensus.cc:738] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f146b4127824a07a5ce867d82511cfe, State: Initialized, Role: FOLLOWER
I20250111 09:02:39.970199 22108 consensus_queue.cc:260] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f146b4127824a07a5ce867d82511cfe" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42633 } }
I20250111 09:02:39.969986 21945 catalog_manager.cc:5526] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f146b4127824a07a5ce867d82511cfe (127.14.61.1). New cstate: current_term: 1 leader_uuid: "7f146b4127824a07a5ce867d82511cfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f146b4127824a07a5ce867d82511cfe" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42633 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:39.970274 22108 raft_consensus.cc:397] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:02:39.970311 22108 raft_consensus.cc:491] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:02:39.970355 22108 raft_consensus.cc:3054] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:39.971117 22108 raft_consensus.cc:513] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f146b4127824a07a5ce867d82511cfe" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42633 } }
I20250111 09:02:39.971232 22108 leader_election.cc:304] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f146b4127824a07a5ce867d82511cfe; no voters:
I20250111 09:02:39.971395 22110 raft_consensus.cc:2798] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:39.971470 22110 raft_consensus.cc:695] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe [term 1 LEADER]: Becoming Leader. State: Replica: 7f146b4127824a07a5ce867d82511cfe, State: Running, Role: LEADER
I20250111 09:02:39.971570 22110 consensus_queue.cc:237] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f146b4127824a07a5ce867d82511cfe" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42633 } }
I20250111 09:02:39.971853 22108 leader_election.cc:290] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:02:39.972070 22108 ts_tablet_manager.cc:1428] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250111 09:02:39.973170 21944 catalog_manager.cc:5526] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f146b4127824a07a5ce867d82511cfe (127.14.61.1). New cstate: current_term: 1 leader_uuid: "7f146b4127824a07a5ce867d82511cfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f146b4127824a07a5ce867d82511cfe" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42633 } health_report { overall_health: HEALTHY } } }
W20250111 09:02:39.980355 22101 txn_manager_proxy_rpc.cc:150] re-attempting BeginTransaction request to one of TxnManagers
I20250111 09:02:40.917766 21944 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57672:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:02:40.922159 22049 tablet_service.cc:1467] Processing CreateTablet for tablet f12591f7b84246d6b5ed34766873c783 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cd79a85692214e0a81b6ed7dbd4e7827]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:02:40.922427 22049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f12591f7b84246d6b5ed34766873c783. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:40.924403 22126 tablet_bootstrap.cc:492] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe: Bootstrap starting.
I20250111 09:02:40.925271 22126 tablet_bootstrap.cc:654] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:40.926105 22126 tablet_bootstrap.cc:492] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe: No bootstrap required, opened a new log
I20250111 09:02:40.926187 22126 ts_tablet_manager.cc:1397] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:02:40.926524 22126 raft_consensus.cc:357] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f146b4127824a07a5ce867d82511cfe" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42633 } }
I20250111 09:02:40.926599 22126 raft_consensus.cc:383] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:40.926633 22126 raft_consensus.cc:738] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f146b4127824a07a5ce867d82511cfe, State: Initialized, Role: FOLLOWER
I20250111 09:02:40.926743 22126 consensus_queue.cc:260] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f146b4127824a07a5ce867d82511cfe" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42633 } }
I20250111 09:02:40.926815 22126 raft_consensus.cc:397] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:02:40.926854 22126 raft_consensus.cc:491] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:02:40.926898 22126 raft_consensus.cc:3054] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:40.927505 22126 raft_consensus.cc:513] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f146b4127824a07a5ce867d82511cfe" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42633 } }
I20250111 09:02:40.927628 22126 leader_election.cc:304] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f146b4127824a07a5ce867d82511cfe; no voters:
I20250111 09:02:40.927781 22126 leader_election.cc:290] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:02:40.927901 22128 raft_consensus.cc:2798] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:40.928040 22128 raft_consensus.cc:695] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe [term 1 LEADER]: Becoming Leader. State: Replica: 7f146b4127824a07a5ce867d82511cfe, State: Running, Role: LEADER
I20250111 09:02:40.928076 22126 ts_tablet_manager.cc:1428] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:40.928143 22128 consensus_queue.cc:237] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f146b4127824a07a5ce867d82511cfe" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42633 } }
I20250111 09:02:40.928458 22129 tablet_replica.cc:440] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f146b4127824a07a5ce867d82511cfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f146b4127824a07a5ce867d82511cfe" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42633 } } }
I20250111 09:02:40.928560 22129 tablet_replica.cc:443] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:02:40.928539 22130 tablet_replica.cc:440] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe: TxnStatusTablet state changed. Reason: New leader 7f146b4127824a07a5ce867d82511cfe. Latest consensus state: current_term: 1 leader_uuid: "7f146b4127824a07a5ce867d82511cfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f146b4127824a07a5ce867d82511cfe" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42633 } } }
I20250111 09:02:40.928609 22130 tablet_replica.cc:443] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:02:40.928768 22132 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:02:40.929059 22132 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:02:40.929273 21944 catalog_manager.cc:5526] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f146b4127824a07a5ce867d82511cfe (127.14.61.1). New cstate: current_term: 1 leader_uuid: "7f146b4127824a07a5ce867d82511cfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f146b4127824a07a5ce867d82511cfe" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42633 } health_report { overall_health: HEALTHY } } }
W20250111 09:02:41.042428 22146 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations
I20250111 09:02:41.046378 14580 tablet_server.cc:178] TabletServer@127.14.61.1:0 shutting down...
I20250111 09:02:41.050261 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250111 09:02:41.050541 14580 tablet_replica.cc:331] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe: stopping tablet replica
I20250111 09:02:41.050753 14580 raft_consensus.cc:2238] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:02:41.050909 14580 raft_consensus.cc:2267] T f12591f7b84246d6b5ed34766873c783 P 7f146b4127824a07a5ce867d82511cfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:02:41.051833 14580 tablet_replica.cc:331] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe: stopping tablet replica
I20250111 09:02:41.051924 14580 raft_consensus.cc:2238] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:02:41.052007 14580 raft_consensus.cc:2267] T 11a48dd21ace4218a5f46fb20805d46d P 7f146b4127824a07a5ce867d82511cfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:02:41.052405 14580 tablet_replica.cc:331] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe: stopping tablet replica
I20250111 09:02:41.052479 14580 raft_consensus.cc:2238] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:02:41.052559 14580 raft_consensus.cc:2267] T 90760889537647fd81890edf2ae2f5cb P 7f146b4127824a07a5ce867d82511cfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:02:41.066205 14580 tablet_server.cc:195] TabletServer@127.14.61.1:0 shutdown complete.
I20250111 09:02:41.069746 14580 master.cc:537] Master@127.14.61.62:33405 shutting down...
I20250111 09:02:41.074637 14580 raft_consensus.cc:2238] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:02:41.074731 14580 raft_consensus.cc:2267] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:02:41.074788 14580 tablet_replica.cc:331] T 00000000000000000000000000000000 P 0a42f82d46734a2a8e36424c339e42d1: stopping tablet replica
I20250111 09:02:41.086916 14580 master.cc:559] Master@127.14.61.62:33405 shutdown complete.
[ OK ] TxnOpDispatcherITest.LifecycleBasic (1269 ms)
[ RUN ] TxnOpDispatcherITest.TestRetryWaitDieAbortsWhenTServerUnavailable
I20250111 09:02:41.092211 14580 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.61.62:44665
I20250111 09:02:41.092473 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:41.094545 22148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:41.095122 22149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:41.095829 22151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:41.096537 14580 server_base.cc:1034] running on GCE node
I20250111 09:02:41.096647 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:02:41.096676 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:02:41.096688 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586161096688 us; error 0 us; skew 500 ppm
I20250111 09:02:41.097494 14580 webserver.cc:458] Webserver started at http://127.14.61.62:43505/ using document root <none> and password file <none>
I20250111 09:02:41.097633 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:41.097676 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:41.097725 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:02:41.098026 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestRetryWaitDieAbortsWhenTServerUnavailable.1736586070622223-14580-0/minicluster-data/master-0-root/instance:
uuid: "f887e74b1cbd4bd89714962b3592388e"
format_stamp: "Formatted at 2025-01-11 09:02:41 on dist-test-slave-nvbt"
I20250111 09:02:41.099258 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250111 09:02:41.099978 22156 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:41.100243 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:02:41.100304 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestRetryWaitDieAbortsWhenTServerUnavailable.1736586070622223-14580-0/minicluster-data/master-0-root
uuid: "f887e74b1cbd4bd89714962b3592388e"
format_stamp: "Formatted at 2025-01-11 09:02:41 on dist-test-slave-nvbt"
I20250111 09:02:41.100354 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestRetryWaitDieAbortsWhenTServerUnavailable.1736586070622223-14580-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestRetryWaitDieAbortsWhenTServerUnavailable.1736586070622223-14580-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestRetryWaitDieAbortsWhenTServerUnavailable.1736586070622223-14580-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:41.116175 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:41.116377 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:41.120097 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:44665
I20250111 09:02:41.121382 22218 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:44665 every 8 connection(s)
I20250111 09:02:41.121713 22219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:41.123009 22219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e: Bootstrap starting.
I20250111 09:02:41.123643 22219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:41.124389 22219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e: No bootstrap required, opened a new log
I20250111 09:02:41.124680 22219 raft_consensus.cc:357] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f887e74b1cbd4bd89714962b3592388e" member_type: VOTER }
I20250111 09:02:41.124743 22219 raft_consensus.cc:383] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:41.124766 22219 raft_consensus.cc:738] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f887e74b1cbd4bd89714962b3592388e, State: Initialized, Role: FOLLOWER
I20250111 09:02:41.124843 22219 consensus_queue.cc:260] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f887e74b1cbd4bd89714962b3592388e" member_type: VOTER }
I20250111 09:02:41.124887 22219 raft_consensus.cc:397] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:02:41.124910 22219 raft_consensus.cc:491] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:02:41.124939 22219 raft_consensus.cc:3054] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:41.125521 22219 raft_consensus.cc:513] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f887e74b1cbd4bd89714962b3592388e" member_type: VOTER }
I20250111 09:02:41.125617 22219 leader_election.cc:304] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f887e74b1cbd4bd89714962b3592388e; no voters:
I20250111 09:02:41.125741 22219 leader_election.cc:290] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:02:41.125847 22222 raft_consensus.cc:2798] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:41.126010 22222 raft_consensus.cc:695] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e [term 1 LEADER]: Becoming Leader. State: Replica: f887e74b1cbd4bd89714962b3592388e, State: Running, Role: LEADER
I20250111 09:02:41.126145 22219 sys_catalog.cc:564] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:02:41.126134 22222 consensus_queue.cc:237] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f887e74b1cbd4bd89714962b3592388e" member_type: VOTER }
I20250111 09:02:41.126574 22223 sys_catalog.cc:455] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f887e74b1cbd4bd89714962b3592388e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f887e74b1cbd4bd89714962b3592388e" member_type: VOTER } }
I20250111 09:02:41.126596 22224 sys_catalog.cc:455] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f887e74b1cbd4bd89714962b3592388e. Latest consensus state: current_term: 1 leader_uuid: "f887e74b1cbd4bd89714962b3592388e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f887e74b1cbd4bd89714962b3592388e" member_type: VOTER } }
I20250111 09:02:41.126655 22223 sys_catalog.cc:458] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e [sys.catalog]: This master's current role is: LEADER
I20250111 09:02:41.126672 22224 sys_catalog.cc:458] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e [sys.catalog]: This master's current role is: LEADER
I20250111 09:02:41.126895 22227 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:02:41.127698 22227 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:02:41.127894 14580 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250111 09:02:41.129230 22227 catalog_manager.cc:1348] Generated new cluster ID: e01698c5c5d14463b4ac102309024f71
I20250111 09:02:41.129271 22227 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:02:41.138264 22227 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:02:41.138655 22227 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:02:41.145061 22227 catalog_manager.cc:5899] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e: Generated new TSK 0
I20250111 09:02:41.145203 22227 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:02:41.148607 22173 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60572:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:02:41.159909 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:41.161756 22246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:41.162360 22247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:41.162931 22249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:41.163503 14580 server_base.cc:1034] running on GCE node
I20250111 09:02:41.163625 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:02:41.163658 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:02:41.163677 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586161163676 us; error 0 us; skew 500 ppm
I20250111 09:02:41.164366 14580 webserver.cc:458] Webserver started at http://127.14.61.1:39727/ using document root <none> and password file <none>
I20250111 09:02:41.164484 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:41.164532 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:41.164595 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:02:41.164876 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestRetryWaitDieAbortsWhenTServerUnavailable.1736586070622223-14580-0/minicluster-data/ts-0-root/instance:
uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa"
format_stamp: "Formatted at 2025-01-11 09:02:41 on dist-test-slave-nvbt"
I20250111 09:02:41.166153 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20250111 09:02:41.166899 22254 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:41.167081 14580 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250111 09:02:41.167135 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestRetryWaitDieAbortsWhenTServerUnavailable.1736586070622223-14580-0/minicluster-data/ts-0-root
uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa"
format_stamp: "Formatted at 2025-01-11 09:02:41 on dist-test-slave-nvbt"
I20250111 09:02:41.167192 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestRetryWaitDieAbortsWhenTServerUnavailable.1736586070622223-14580-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestRetryWaitDieAbortsWhenTServerUnavailable.1736586070622223-14580-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestRetryWaitDieAbortsWhenTServerUnavailable.1736586070622223-14580-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:41.190054 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:41.190331 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:41.190933 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:02:41.190971 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:41.191020 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:02:41.191057 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:41.195956 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:44609
I20250111 09:02:41.195966 22324 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:44609 every 8 connection(s)
I20250111 09:02:41.196702 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250111 09:02:41.196866 22325 heartbeater.cc:346] Connected to a master server at 127.14.61.62:44665
I20250111 09:02:41.196941 22325 heartbeater.cc:463] Registering TS with master...
I20250111 09:02:41.197129 22325 heartbeater.cc:510] Master 127.14.61.62:44665 requested a full tablet report, sending...
I20250111 09:02:41.197587 22173 ts_manager.cc:194] Registered new tserver with Master: a20f8c5ea9bc4dc4ab509b1d437485aa (127.14.61.1:44609)
I20250111 09:02:41.198390 22173 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60582
W20250111 09:02:41.198822 22330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:41.199358 22328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:41.200012 14580 server_base.cc:1034] running on GCE node
W20250111 09:02:41.200477 22327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:41.200769 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:02:41.200824 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:02:41.200845 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586161200844 us; error 0 us; skew 500 ppm
I20250111 09:02:41.201606 14580 webserver.cc:458] Webserver started at http://127.14.61.2:36835/ using document root <none> and password file <none>
I20250111 09:02:41.201719 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:41.201762 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:41.201822 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:02:41.202126 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestRetryWaitDieAbortsWhenTServerUnavailable.1736586070622223-14580-0/minicluster-data/ts-1-root/instance:
uuid: "cd293301254647f08621baa54810b5cb"
format_stamp: "Formatted at 2025-01-11 09:02:41 on dist-test-slave-nvbt"
I20250111 09:02:41.203266 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250111 09:02:41.203950 22335 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:41.204182 14580 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250111 09:02:41.204241 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestRetryWaitDieAbortsWhenTServerUnavailable.1736586070622223-14580-0/minicluster-data/ts-1-root
uuid: "cd293301254647f08621baa54810b5cb"
format_stamp: "Formatted at 2025-01-11 09:02:41 on dist-test-slave-nvbt"
I20250111 09:02:41.204298 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestRetryWaitDieAbortsWhenTServerUnavailable.1736586070622223-14580-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestRetryWaitDieAbortsWhenTServerUnavailable.1736586070622223-14580-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestRetryWaitDieAbortsWhenTServerUnavailable.1736586070622223-14580-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:41.208498 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:41.208748 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:41.209367 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:02:41.209412 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:41.209460 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:02:41.209493 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:41.214830 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.2:38151
I20250111 09:02:41.214859 22405 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.2:38151 every 8 connection(s)
I20250111 09:02:41.215649 22406 heartbeater.cc:346] Connected to a master server at 127.14.61.62:44665
I20250111 09:02:41.215724 22406 heartbeater.cc:463] Registering TS with master...
I20250111 09:02:41.215878 22406 heartbeater.cc:510] Master 127.14.61.62:44665 requested a full tablet report, sending...
I20250111 09:02:41.216351 22173 ts_manager.cc:194] Registered new tserver with Master: cd293301254647f08621baa54810b5cb (127.14.61.2:38151)
I20250111 09:02:41.216460 14580 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.001231365s
I20250111 09:02:41.216857 22173 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60602
I20250111 09:02:42.155653 22173 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60624:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:02:42.162653 22284 tablet_service.cc:1467] Processing CreateTablet for tablet e02b358e69a74c99a360a93551936a79 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4e8bfd27d4f44b7e96d27291dacb98ab]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:02:42.162911 22284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e02b358e69a74c99a360a93551936a79. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:42.164592 22426 tablet_bootstrap.cc:492] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa: Bootstrap starting.
I20250111 09:02:42.165386 22426 tablet_bootstrap.cc:654] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:42.166395 22426 tablet_bootstrap.cc:492] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa: No bootstrap required, opened a new log
I20250111 09:02:42.166458 22426 ts_tablet_manager.cc:1397] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:02:42.166744 22426 raft_consensus.cc:357] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44609 } }
I20250111 09:02:42.166814 22426 raft_consensus.cc:383] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:42.166838 22426 raft_consensus.cc:738] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a20f8c5ea9bc4dc4ab509b1d437485aa, State: Initialized, Role: FOLLOWER
I20250111 09:02:42.166930 22426 consensus_queue.cc:260] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44609 } }
I20250111 09:02:42.166993 22426 raft_consensus.cc:397] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:02:42.167019 22426 raft_consensus.cc:491] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:02:42.167049 22426 raft_consensus.cc:3054] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:42.167631 22426 raft_consensus.cc:513] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44609 } }
I20250111 09:02:42.167727 22426 leader_election.cc:304] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a20f8c5ea9bc4dc4ab509b1d437485aa; no voters:
I20250111 09:02:42.167881 22426 leader_election.cc:290] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:02:42.167979 22428 raft_consensus.cc:2798] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:42.168138 22428 raft_consensus.cc:695] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [term 1 LEADER]: Becoming Leader. State: Replica: a20f8c5ea9bc4dc4ab509b1d437485aa, State: Running, Role: LEADER
I20250111 09:02:42.168174 22325 heartbeater.cc:502] Master 127.14.61.62:44665 was elected leader, sending a full tablet report...
I20250111 09:02:42.168265 22428 consensus_queue.cc:237] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44609 } }
I20250111 09:02:42.168161 22426 ts_tablet_manager.cc:1428] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:42.168586 22429 tablet_replica.cc:440] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44609 } } }
I20250111 09:02:42.168835 22430 tablet_replica.cc:440] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa: TxnStatusTablet state changed. Reason: New leader a20f8c5ea9bc4dc4ab509b1d437485aa. Latest consensus state: current_term: 1 leader_uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44609 } } }
I20250111 09:02:42.169121 22430 tablet_replica.cc:443] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:02:42.168758 22429 tablet_replica.cc:443] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:02:42.169364 22433 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:02:42.169442 22433 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:02:42.169454 22172 catalog_manager.cc:5526] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa reported cstate change: term changed from 0 to 1, leader changed from <none> to a20f8c5ea9bc4dc4ab509b1d437485aa (127.14.61.1). New cstate: current_term: 1 leader_uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44609 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:42.217567 22406 heartbeater.cc:502] Master 127.14.61.62:44665 was elected leader, sending a full tablet report...
I20250111 09:02:42.246758 22172 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60622:
name: "txn_op_dispatcher_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250111 09:02:42.253736 22364 tablet_service.cc:1467] Processing CreateTablet for tablet 4d9aea64824443f9991e7ec3f78cf821 (DEFAULT_TABLE table=txn_op_dispatcher_test [id=d2381bbc73fa4127bbecd76c65574cab]), partition=RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:42.253968 22364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d9aea64824443f9991e7ec3f78cf821. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:42.255735 22435 tablet_bootstrap.cc:492] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb: Bootstrap starting.
I20250111 09:02:42.256533 22435 tablet_bootstrap.cc:654] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:42.257375 22435 tablet_bootstrap.cc:492] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb: No bootstrap required, opened a new log
I20250111 09:02:42.257452 22435 ts_tablet_manager.cc:1397] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:02:42.257781 22435 raft_consensus.cc:357] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd293301254647f08621baa54810b5cb" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38151 } }
I20250111 09:02:42.257854 22435 raft_consensus.cc:383] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:42.257887 22435 raft_consensus.cc:738] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd293301254647f08621baa54810b5cb, State: Initialized, Role: FOLLOWER
I20250111 09:02:42.258001 22435 consensus_queue.cc:260] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd293301254647f08621baa54810b5cb" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38151 } }
I20250111 09:02:42.258064 22435 raft_consensus.cc:397] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:02:42.258103 22435 raft_consensus.cc:491] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:02:42.258150 22435 raft_consensus.cc:3054] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:42.258721 22435 raft_consensus.cc:513] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd293301254647f08621baa54810b5cb" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38151 } }
I20250111 09:02:42.258828 22435 leader_election.cc:304] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd293301254647f08621baa54810b5cb; no voters:
I20250111 09:02:42.259011 22435 leader_election.cc:290] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:02:42.259166 22437 raft_consensus.cc:2798] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:42.259361 22435 ts_tablet_manager.cc:1428] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:42.259388 22437 raft_consensus.cc:695] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb [term 1 LEADER]: Becoming Leader. State: Replica: cd293301254647f08621baa54810b5cb, State: Running, Role: LEADER
I20250111 09:02:42.259600 22437 consensus_queue.cc:237] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd293301254647f08621baa54810b5cb" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38151 } }
I20250111 09:02:42.260759 22172 catalog_manager.cc:5526] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb reported cstate change: term changed from 0 to 1, leader changed from <none> to cd293301254647f08621baa54810b5cb (127.14.61.2). New cstate: current_term: 1 leader_uuid: "cd293301254647f08621baa54810b5cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd293301254647f08621baa54810b5cb" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 38151 } health_report { overall_health: HEALTHY } } }
W20250111 09:02:43.287751 22441 lock_manager.cc:494] Txn 1 has not acquired the partition lock after 1000ms
W20250111 09:02:43.288110 22441 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:50980: Aborted: Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID
I20250111 09:02:43.288182 22450 ts_tablet_manager.cc:2109] Sending abort request for transaction 1
I20250111 09:02:43.288353 22441 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50980 (ReqId={client: e1e8db34afa44ffc9956ec1077993a8c, seq_no=1, attempt_no=0}) took 1004 ms. Trace:
I20250111 09:02:43.288420 22441 rpcz_store.cc:276] 0111 09:02:42.284134 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:42.284186 (+ 52us) service_pool.cc:225] Handling call
0111 09:02:43.288339 (+1004153us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:42.286247 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:42.286285 (+ 38us) participant_op.cc:199] PREPARE: finished
0111 09:02:42.286298 (+ 13us) participant_op.cc:212] START. Timestamp: 1736586162286292
0111 09:02:42.286299 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:02:42.286374 (+ 75us) log.cc:844] Serialized 65 byte log entry
0111 09:02:42.286686 (+ 312us) op_driver.cc:464] REPLICATION: finished
0111 09:02:42.286762 (+ 76us) participant_op.cc:265] APPLY: starting
0111 09:02:42.286871 (+ 109us) participant_op.cc:270] APPLY: finished
0111 09:02:42.286908 (+ 37us) log.cc:844] Serialized 14 byte log entry
0111 09:02:42.286933 (+ 25us) participant_op.cc:94] Released txn lock
0111 09:02:42.286939 (+ 6us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:02:42.287230 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 4d9aea64824443f9991e7ec3f78cf821
0111 09:02:42.287300 (+ 70us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:42.287301 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:02:42.287302 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:42.287335 (+ 33us) write_op.cc:620] Acquiring the partition lock for write op
0111 09:02:43.287885 (+1000550us) op_driver.cc:389] HandleFailure(Aborted: Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID)
0111 09:02:43.287898 (+ 13us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:43.287903 (+ 5us) write_op.cc:454] Released schema lock
0111 09:02:43.287908 (+ 5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":28,"txn-participant-registration.run_cpu_time_us":276,"txn-participant-registration.run_wall_time_us":1854,"child_traces":[["op",{"apply.queue_time_us":39,"apply.run_cpu_time_us":340,"apply.run_wall_time_us":340,"prepare.queue_time_us":35,"prepare.run_cpu_time_us":158,"prepare.run_wall_time_us":158,"replication_time_us":367,"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":161,"thread_start_us":81,"threads_started":1,"txn-participant-registration.queue_time_us":149}]]}]]}
W20250111 09:02:43.289495 22410 batcher.cc:441] Aborted: Failed to write batch of 1 ops to tablet 4d9aea64824443f9991e7ec3f78cf821 after 1 attempt(s): Failed to write to server: cd293301254647f08621baa54810b5cb (127.14.61.2:38151): Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID
I20250111 09:02:43.289675 14580 tablet_server.cc:178] TabletServer@127.14.61.1:0 shutting down...
I20250111 09:02:43.293398 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250111 09:02:43.293570 14580 tablet_replica.cc:331] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa: stopping tablet replica
I20250111 09:02:43.293677 14580 raft_consensus.cc:2238] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:02:43.293788 14580 raft_consensus.cc:2267] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [term 1 FOLLOWER]: Raft consensus is shut down!
W20250111 09:02:43.297206 22441 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations [suppressed 1 similar messages]
I20250111 09:02:43.344460 14580 txn_status_manager.cc:765] Waiting for 1 task(s) to stop
I20250111 09:02:43.356696 14580 tablet_server.cc:195] TabletServer@127.14.61.1:0 shutdown complete.
I20250111 09:02:46.359119 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:46.361702 22461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:46.362481 22464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:46.363150 22462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:46.363873 14580 server_base.cc:1034] running on GCE node
I20250111 09:02:46.364012 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:02:46.364061 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:02:46.364087 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586166364086 us; error 0 us; skew 500 ppm
I20250111 09:02:46.370306 14580 webserver.cc:458] Webserver started at http://127.14.61.1:39727/ using document root <none> and password file <none>
I20250111 09:02:46.370462 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:46.370512 14580 fs_manager.cc:365] Using existing metadata directory in first data directory
I20250111 09:02:46.371592 14580 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:02:46.372387 22469 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:46.372622 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:02:46.372675 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestRetryWaitDieAbortsWhenTServerUnavailable.1736586070622223-14580-0/minicluster-data/ts-0-root
uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa"
format_stamp: "Formatted at 2025-01-11 09:02:41 on dist-test-slave-nvbt"
I20250111 09:02:46.372737 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestRetryWaitDieAbortsWhenTServerUnavailable.1736586070622223-14580-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestRetryWaitDieAbortsWhenTServerUnavailable.1736586070622223-14580-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestRetryWaitDieAbortsWhenTServerUnavailable.1736586070622223-14580-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:46.379837 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:46.380076 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:46.381177 22477 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20250111 09:02:46.382853 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20250111 09:02:46.382932 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20250111 09:02:46.382980 14580 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20250111 09:02:46.383919 14580 ts_tablet_manager.cc:610] Registered 1 tablets
I20250111 09:02:46.383967 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20250111 09:02:46.384016 22477 tablet_bootstrap.cc:492] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa: Bootstrap starting.
I20250111 09:02:46.389297 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:44609
I20250111 09:02:46.389320 22542 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:44609 every 8 connection(s)
I20250111 09:02:46.390620 22543 heartbeater.cc:346] Connected to a master server at 127.14.61.62:44665
I20250111 09:02:46.390717 22543 heartbeater.cc:463] Registering TS with master...
I20250111 09:02:46.390899 22543 heartbeater.cc:510] Master 127.14.61.62:44665 requested a full tablet report, sending...
I20250111 09:02:46.391724 22477 tablet_bootstrap.cc:492] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=4 ignored=0} mutations{seen=1 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250111 09:02:46.391842 22173 ts_manager.cc:194] Re-registered known tserver with Master: a20f8c5ea9bc4dc4ab509b1d437485aa (127.14.61.1:44609)
I20250111 09:02:46.392148 22477 tablet_bootstrap.cc:492] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa: Bootstrap complete.
I20250111 09:02:46.392290 22477 ts_tablet_manager.cc:1397] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20250111 09:02:46.392666 22173 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36826
I20250111 09:02:46.392659 22477 raft_consensus.cc:357] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44609 } }
I20250111 09:02:46.392756 22477 raft_consensus.cc:738] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a20f8c5ea9bc4dc4ab509b1d437485aa, State: Initialized, Role: FOLLOWER
I20250111 09:02:46.392935 22477 consensus_queue.cc:260] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44609 } }
I20250111 09:02:46.393054 22477 raft_consensus.cc:397] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:02:46.393095 22477 raft_consensus.cc:491] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:02:46.393141 22477 raft_consensus.cc:3054] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [term 1 FOLLOWER]: Advancing to term 2
I20250111 09:02:46.393975 22477 raft_consensus.cc:513] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44609 } }
I20250111 09:02:46.394102 22477 leader_election.cc:304] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a20f8c5ea9bc4dc4ab509b1d437485aa; no voters:
I20250111 09:02:46.394270 22477 leader_election.cc:290] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [CANDIDATE]: Term 2 election: Requested vote from peers
I20250111 09:02:46.394379 22547 raft_consensus.cc:2798] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [term 2 FOLLOWER]: Leader election won for term 2
I20250111 09:02:46.394641 22547 raft_consensus.cc:695] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [term 2 LEADER]: Becoming Leader. State: Replica: a20f8c5ea9bc4dc4ab509b1d437485aa, State: Running, Role: LEADER
I20250111 09:02:46.394779 22547 consensus_queue.cc:237] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44609 } }
I20250111 09:02:46.394820 22477 ts_tablet_manager.cc:1428] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:46.394825 22543 heartbeater.cc:502] Master 127.14.61.62:44665 was elected leader, sending a full tablet report...
I20250111 09:02:46.395072 22549 tablet_replica.cc:440] TxnStatusTablet state changed. Reason: New leader a20f8c5ea9bc4dc4ab509b1d437485aa. Latest consensus state: current_term: 2 leader_uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44609 } } }
I20250111 09:02:46.395092 22548 tablet_replica.cc:440] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44609 } } }
I20250111 09:02:46.395143 22549 tablet_replica.cc:443] This TxnStatusTablet replica's current role is: LEADER
I20250111 09:02:46.395166 22548 tablet_replica.cc:443] This TxnStatusTablet replica's current role is: LEADER
I20250111 09:02:46.395437 22551 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:02:46.395707 22551 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:02:46.395797 22173 catalog_manager.cc:5526] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20f8c5ea9bc4dc4ab509b1d437485aa" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44609 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:46.396932 22551 txn_status_manager.cc:728] Starting 1 aborts task
W20250111 09:02:46.401314 22559 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34548: Illegal state: Transaction metadata for transaction 1 already set
I20250111 09:02:46.406522 14580 tablet_server.cc:178] TabletServer@127.14.61.1:44609 shutting down...
I20250111 09:02:46.409294 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250111 09:02:46.409488 14580 tablet_replica.cc:331] stopping tablet replica
I20250111 09:02:46.409595 14580 raft_consensus.cc:2238] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [term 2 LEADER]: Raft consensus shutting down.
I20250111 09:02:46.409693 14580 raft_consensus.cc:2267] T e02b358e69a74c99a360a93551936a79 P a20f8c5ea9bc4dc4ab509b1d437485aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20250111 09:02:46.422250 14580 tablet_server.cc:195] TabletServer@127.14.61.1:44609 shutdown complete.
I20250111 09:02:46.424211 14580 tablet_server.cc:178] TabletServer@127.14.61.2:0 shutting down...
I20250111 09:02:46.427754 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250111 09:02:46.427879 14580 tablet_replica.cc:331] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb: stopping tablet replica
I20250111 09:02:46.427975 14580 raft_consensus.cc:2238] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:02:46.428071 14580 raft_consensus.cc:2267] T 4d9aea64824443f9991e7ec3f78cf821 P cd293301254647f08621baa54810b5cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:02:46.439842 14580 tablet_server.cc:195] TabletServer@127.14.61.2:0 shutdown complete.
I20250111 09:02:46.441599 14580 master.cc:537] Master@127.14.61.62:44665 shutting down...
I20250111 09:02:46.444684 14580 raft_consensus.cc:2238] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:02:46.444783 14580 raft_consensus.cc:2267] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:02:46.444835 14580 tablet_replica.cc:331] T 00000000000000000000000000000000 P f887e74b1cbd4bd89714962b3592388e: stopping tablet replica
I20250111 09:02:46.457052 14580 master.cc:559] Master@127.14.61.62:44665 shutdown complete.
[ OK ] TxnOpDispatcherITest.TestRetryWaitDieAbortsWhenTServerUnavailable (5369 ms)
[ RUN ] TxnOpDispatcherITest.BeginTxnLockAbort
I20250111 09:02:46.461884 14580 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.61.62:34079
I20250111 09:02:46.462129 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:46.464151 22562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:46.464783 22565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:46.465443 22563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:46.466198 14580 server_base.cc:1034] running on GCE node
I20250111 09:02:46.466356 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:02:46.466387 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:02:46.466406 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586166466405 us; error 0 us; skew 500 ppm
I20250111 09:02:46.467065 14580 webserver.cc:458] Webserver started at http://127.14.61.62:43899/ using document root <none> and password file <none>
I20250111 09:02:46.467186 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:46.467247 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:46.467309 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:02:46.467604 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.BeginTxnLockAbort.1736586070622223-14580-0/minicluster-data/master-0-root/instance:
uuid: "92b01fd1eb4f4112aa2067e6764c66c2"
format_stamp: "Formatted at 2025-01-11 09:02:46 on dist-test-slave-nvbt"
I20250111 09:02:46.468737 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:02:46.469458 22570 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:46.469623 14580 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:46.469676 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.BeginTxnLockAbort.1736586070622223-14580-0/minicluster-data/master-0-root
uuid: "92b01fd1eb4f4112aa2067e6764c66c2"
format_stamp: "Formatted at 2025-01-11 09:02:46 on dist-test-slave-nvbt"
I20250111 09:02:46.469730 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.BeginTxnLockAbort.1736586070622223-14580-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.BeginTxnLockAbort.1736586070622223-14580-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.BeginTxnLockAbort.1736586070622223-14580-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:46.475679 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:46.475888 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:46.479358 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:34079
I20250111 09:02:46.480166 22632 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:34079 every 8 connection(s)
I20250111 09:02:46.480476 22633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:46.481981 22633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2: Bootstrap starting.
I20250111 09:02:46.482586 22633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:46.483299 22633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2: No bootstrap required, opened a new log
I20250111 09:02:46.483579 22633 raft_consensus.cc:357] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b01fd1eb4f4112aa2067e6764c66c2" member_type: VOTER }
I20250111 09:02:46.483647 22633 raft_consensus.cc:383] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:46.483678 22633 raft_consensus.cc:738] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92b01fd1eb4f4112aa2067e6764c66c2, State: Initialized, Role: FOLLOWER
I20250111 09:02:46.483785 22633 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b01fd1eb4f4112aa2067e6764c66c2" member_type: VOTER }
I20250111 09:02:46.483855 22633 raft_consensus.cc:397] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:02:46.483891 22633 raft_consensus.cc:491] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:02:46.483935 22633 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:46.484463 22633 raft_consensus.cc:513] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b01fd1eb4f4112aa2067e6764c66c2" member_type: VOTER }
I20250111 09:02:46.484568 22633 leader_election.cc:304] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92b01fd1eb4f4112aa2067e6764c66c2; no voters:
I20250111 09:02:46.484704 22633 leader_election.cc:290] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:02:46.484783 22636 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:46.484953 22636 raft_consensus.cc:695] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2 [term 1 LEADER]: Becoming Leader. State: Replica: 92b01fd1eb4f4112aa2067e6764c66c2, State: Running, Role: LEADER
I20250111 09:02:46.485050 22633 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2 [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:02:46.485080 22636 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b01fd1eb4f4112aa2067e6764c66c2" member_type: VOTER }
I20250111 09:02:46.485445 22637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92b01fd1eb4f4112aa2067e6764c66c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b01fd1eb4f4112aa2067e6764c66c2" member_type: VOTER } }
I20250111 09:02:46.485499 22638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92b01fd1eb4f4112aa2067e6764c66c2. Latest consensus state: current_term: 1 leader_uuid: "92b01fd1eb4f4112aa2067e6764c66c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b01fd1eb4f4112aa2067e6764c66c2" member_type: VOTER } }
I20250111 09:02:46.485620 22638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2 [sys.catalog]: This master's current role is: LEADER
I20250111 09:02:46.485605 22637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2 [sys.catalog]: This master's current role is: LEADER
I20250111 09:02:46.486058 22642 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:02:46.486792 22642 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:02:46.486987 14580 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250111 09:02:46.488530 22642 catalog_manager.cc:1348] Generated new cluster ID: 1e2206e7496f4a9682fcf4cc0ecff3fd
I20250111 09:02:46.488576 22642 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:02:46.504428 22642 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:02:46.504840 22642 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:02:46.516793 22642 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2: Generated new TSK 0
I20250111 09:02:46.516922 22642 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:02:46.519013 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:46.521200 22661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:46.521785 22660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:46.522544 22663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:46.523121 14580 server_base.cc:1034] running on GCE node
I20250111 09:02:46.523252 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:02:46.523300 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:02:46.523315 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586166523315 us; error 0 us; skew 500 ppm
I20250111 09:02:46.524071 14580 webserver.cc:458] Webserver started at http://127.14.61.1:34415/ using document root <none> and password file <none>
I20250111 09:02:46.524179 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:46.524214 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:46.524263 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:02:46.524597 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.BeginTxnLockAbort.1736586070622223-14580-0/minicluster-data/ts-0-root/instance:
uuid: "44cd71e01cd641bea1fafcb1fb7633d6"
format_stamp: "Formatted at 2025-01-11 09:02:46 on dist-test-slave-nvbt"
I20250111 09:02:46.526132 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250111 09:02:46.526964 22668 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:46.527055 22587 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43218:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:02:46.527163 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20250111 09:02:46.527210 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.BeginTxnLockAbort.1736586070622223-14580-0/minicluster-data/ts-0-root
uuid: "44cd71e01cd641bea1fafcb1fb7633d6"
format_stamp: "Formatted at 2025-01-11 09:02:46 on dist-test-slave-nvbt"
I20250111 09:02:46.527261 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.BeginTxnLockAbort.1736586070622223-14580-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.BeginTxnLockAbort.1736586070622223-14580-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.BeginTxnLockAbort.1736586070622223-14580-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:46.539685 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:46.539918 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:46.540516 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:02:46.540593 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:46.540647 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:02:46.540750 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:46.546418 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:34487
I20250111 09:02:46.546453 22738 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:34487 every 8 connection(s)
I20250111 09:02:46.547345 22739 heartbeater.cc:346] Connected to a master server at 127.14.61.62:34079
I20250111 09:02:46.547801 22739 heartbeater.cc:463] Registering TS with master...
I20250111 09:02:46.547999 22739 heartbeater.cc:510] Master 127.14.61.62:34079 requested a full tablet report, sending...
I20250111 09:02:46.548627 22586 ts_manager.cc:194] Registered new tserver with Master: 44cd71e01cd641bea1fafcb1fb7633d6 (127.14.61.1:34487)
I20250111 09:02:46.549206 14580 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002395883s
I20250111 09:02:46.549414 22586 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43232
I20250111 09:02:46.554930 22586 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43246:
name: "txn_op_dispatcher_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
I20250111 09:02:46.562300 22697 tablet_service.cc:1467] Processing CreateTablet for tablet a0634746f88040d7990a2440b83c7575 (DEFAULT_TABLE table=txn_op_dispatcher_test [id=dcfa943e461144bba0ff6598087dd9b2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:46.562300 22698 tablet_service.cc:1467] Processing CreateTablet for tablet 7bd9036707de43dbb1f76c1f97a24b68 (DEFAULT_TABLE table=txn_op_dispatcher_test [id=dcfa943e461144bba0ff6598087dd9b2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:46.562598 22697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0634746f88040d7990a2440b83c7575. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:46.562760 22698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bd9036707de43dbb1f76c1f97a24b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:46.564901 22749 tablet_bootstrap.cc:492] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6: Bootstrap starting.
I20250111 09:02:46.565596 22749 tablet_bootstrap.cc:654] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:46.566402 22749 tablet_bootstrap.cc:492] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6: No bootstrap required, opened a new log
I20250111 09:02:46.566466 22749 ts_tablet_manager.cc:1397] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:02:46.566749 22749 raft_consensus.cc:357] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 34487 } }
I20250111 09:02:46.566820 22749 raft_consensus.cc:383] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:46.566850 22749 raft_consensus.cc:738] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44cd71e01cd641bea1fafcb1fb7633d6, State: Initialized, Role: FOLLOWER
I20250111 09:02:46.567001 22749 consensus_queue.cc:260] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 34487 } }
I20250111 09:02:46.567065 22749 raft_consensus.cc:397] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:02:46.567102 22749 raft_consensus.cc:491] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:02:46.567152 22749 raft_consensus.cc:3054] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:46.567901 22749 raft_consensus.cc:513] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 34487 } }
I20250111 09:02:46.568007 22749 leader_election.cc:304] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44cd71e01cd641bea1fafcb1fb7633d6; no voters:
I20250111 09:02:46.568190 22749 leader_election.cc:290] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:02:46.568276 22751 raft_consensus.cc:2798] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:46.568465 22739 heartbeater.cc:502] Master 127.14.61.62:34079 was elected leader, sending a full tablet report...
I20250111 09:02:46.568444 22751 raft_consensus.cc:695] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 1 LEADER]: Becoming Leader. State: Replica: 44cd71e01cd641bea1fafcb1fb7633d6, State: Running, Role: LEADER
I20250111 09:02:46.568450 22749 ts_tablet_manager.cc:1428] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:46.568629 22751 consensus_queue.cc:237] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 34487 } }
I20250111 09:02:46.568687 22749 tablet_bootstrap.cc:492] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6: Bootstrap starting.
I20250111 09:02:46.569603 22749 tablet_bootstrap.cc:654] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:46.569708 22586 catalog_manager.cc:5526] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 44cd71e01cd641bea1fafcb1fb7633d6 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 34487 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:46.570410 22749 tablet_bootstrap.cc:492] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6: No bootstrap required, opened a new log
I20250111 09:02:46.570484 22749 ts_tablet_manager.cc:1397] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:02:46.570902 22749 raft_consensus.cc:357] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 34487 } }
I20250111 09:02:46.571041 22749 raft_consensus.cc:383] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:46.571092 22749 raft_consensus.cc:738] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44cd71e01cd641bea1fafcb1fb7633d6, State: Initialized, Role: FOLLOWER
I20250111 09:02:46.571213 22749 consensus_queue.cc:260] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 34487 } }
I20250111 09:02:46.571285 22749 raft_consensus.cc:397] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:02:46.571357 22749 raft_consensus.cc:491] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:02:46.571413 22749 raft_consensus.cc:3054] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:46.572083 22749 raft_consensus.cc:513] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 34487 } }
I20250111 09:02:46.572181 22749 leader_election.cc:304] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44cd71e01cd641bea1fafcb1fb7633d6; no voters:
I20250111 09:02:46.572263 22749 leader_election.cc:290] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:02:46.572317 22751 raft_consensus.cc:2798] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:46.572443 22751 raft_consensus.cc:695] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 1 LEADER]: Becoming Leader. State: Replica: 44cd71e01cd641bea1fafcb1fb7633d6, State: Running, Role: LEADER
I20250111 09:02:46.572461 22749 ts_tablet_manager.cc:1428] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:46.572576 22751 consensus_queue.cc:237] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 34487 } }
I20250111 09:02:46.573537 22586 catalog_manager.cc:5526] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 44cd71e01cd641bea1fafcb1fb7633d6 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 34487 } health_report { overall_health: HEALTHY } } }
W20250111 09:02:46.577714 22743 txn_manager_proxy_rpc.cc:150] re-attempting BeginTransaction request to one of TxnManagers
I20250111 09:02:47.535257 22586 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43250:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:02:47.539422 22698 tablet_service.cc:1467] Processing CreateTablet for tablet 37c4c9fea8fd4771a9c8f6b4872d0bf7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7243308b1d1541e49907881c9d71cda6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:02:47.539656 22698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37c4c9fea8fd4771a9c8f6b4872d0bf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:47.541385 22767 tablet_bootstrap.cc:492] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6: Bootstrap starting.
I20250111 09:02:47.542171 22767 tablet_bootstrap.cc:654] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:47.542981 22767 tablet_bootstrap.cc:492] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6: No bootstrap required, opened a new log
I20250111 09:02:47.543061 22767 ts_tablet_manager.cc:1397] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:02:47.543373 22767 raft_consensus.cc:357] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 34487 } }
I20250111 09:02:47.543452 22767 raft_consensus.cc:383] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:47.543493 22767 raft_consensus.cc:738] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44cd71e01cd641bea1fafcb1fb7633d6, State: Initialized, Role: FOLLOWER
I20250111 09:02:47.543608 22767 consensus_queue.cc:260] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 34487 } }
I20250111 09:02:47.543681 22767 raft_consensus.cc:397] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:02:47.543722 22767 raft_consensus.cc:491] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:02:47.543771 22767 raft_consensus.cc:3054] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:47.544410 22767 raft_consensus.cc:513] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 34487 } }
I20250111 09:02:47.544529 22767 leader_election.cc:304] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44cd71e01cd641bea1fafcb1fb7633d6; no voters:
I20250111 09:02:47.544706 22767 leader_election.cc:290] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:02:47.544775 22769 raft_consensus.cc:2798] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:47.544941 22769 raft_consensus.cc:695] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 1 LEADER]: Becoming Leader. State: Replica: 44cd71e01cd641bea1fafcb1fb7633d6, State: Running, Role: LEADER
I20250111 09:02:47.545069 22767 ts_tablet_manager.cc:1428] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:47.545161 22769 consensus_queue.cc:237] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 34487 } }
I20250111 09:02:47.545612 22770 tablet_replica.cc:440] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 34487 } } }
I20250111 09:02:47.545720 22771 tablet_replica.cc:440] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6: TxnStatusTablet state changed. Reason: New leader 44cd71e01cd641bea1fafcb1fb7633d6. Latest consensus state: current_term: 1 leader_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 34487 } } }
I20250111 09:02:47.545755 22770 tablet_replica.cc:443] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:02:47.545820 22771 tablet_replica.cc:443] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:02:47.546332 22773 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:02:47.546418 22773 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:02:47.546518 22586 catalog_manager.cc:5526] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 44cd71e01cd641bea1fafcb1fb7633d6 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cd71e01cd641bea1fafcb1fb7633d6" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 34487 } health_report { overall_health: HEALTHY } } }
W20250111 09:02:49.330873 22778 lock_manager.cc:494] Txn 9223372036854775807 has not acquired the partition lock after 1000ms
W20250111 09:02:49.331039 22778 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39942: Aborted: Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID
I20250111 09:02:49.331194 22778 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39942 (ReqId={client: 71793d6b41144abaa40b80539c8414a7, seq_no=6, attempt_no=0}) took 1001 ms. Trace:
I20250111 09:02:49.331230 22778 rpcz_store.cc:276] 0111 09:02:48.330118 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:48.330171 (+ 53us) service_pool.cc:225] Handling call
0111 09:02:49.331184 (+1001013us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:48.330428 (+ 0us) write_op.cc:183] PREPARE: starting on tablet a0634746f88040d7990a2440b83c7575
0111 09:02:48.330481 (+ 53us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:48.330483 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:02:48.330484 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:48.330515 (+ 31us) write_op.cc:620] Acquiring the partition lock for write op
0111 09:02:49.331014 (+1000499us) op_driver.cc:389] HandleFailure(Aborted: Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID)
0111 09:02:49.331026 (+ 12us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:49.331031 (+ 5us) write_op.cc:454] Released schema lock
0111 09:02:49.331036 (+ 5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":44}]]}
W20250111 09:02:49.331670 22744 batcher.cc:441] Aborted: Failed to write batch of 1 ops to tablet a0634746f88040d7990a2440b83c7575 after 1 attempt(s): Failed to write to server: 44cd71e01cd641bea1fafcb1fb7633d6 (127.14.61.1:34487): Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID
W20250111 09:02:50.339583 22778 lock_manager.cc:494] Txn 1 has not acquired the partition lock after 1000ms
W20250111 09:02:50.339869 22778 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39942: Aborted: Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID
I20250111 09:02:50.339947 22793 ts_tablet_manager.cc:2109] Sending abort request for transaction 1
I20250111 09:02:50.340065 22778 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39942 (ReqId={client: 71793d6b41144abaa40b80539c8414a7, seq_no=7, attempt_no=0}) took 1004 ms. Trace:
I20250111 09:02:50.340111 22778 rpcz_store.cc:276] 0111 09:02:49.335619 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:49.335669 (+ 50us) service_pool.cc:225] Handling call
0111 09:02:50.340050 (+1004381us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:49.337949 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:49.337975 (+ 26us) participant_op.cc:199] PREPARE: finished
0111 09:02:49.337992 (+ 17us) participant_op.cc:212] START. Timestamp: 1736586169337984
0111 09:02:49.337993 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:02:49.338066 (+ 73us) log.cc:844] Serialized 65 byte log entry
0111 09:02:49.338527 (+ 461us) op_driver.cc:464] REPLICATION: finished
0111 09:02:49.338617 (+ 90us) participant_op.cc:265] APPLY: starting
0111 09:02:49.338770 (+ 153us) participant_op.cc:270] APPLY: finished
0111 09:02:49.338790 (+ 20us) log.cc:844] Serialized 14 byte log entry
0111 09:02:49.338834 (+ 44us) participant_op.cc:94] Released txn lock
0111 09:02:49.338838 (+ 4us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:02:49.338992 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7bd9036707de43dbb1f76c1f97a24b68
0111 09:02:49.339065 (+ 73us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:49.339066 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:02:49.339067 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:49.339106 (+ 39us) write_op.cc:620] Acquiring the partition lock for write op
0111 09:02:50.339718 (+1000612us) op_driver.cc:389] HandleFailure(Aborted: Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID)
0111 09:02:50.339730 (+ 12us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:50.339735 (+ 5us) write_op.cc:454] Released schema lock
0111 09:02:50.339741 (+ 6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"thread_start_us":56,"threads_started":1,"txn-participant-registration.queue_time_us":106,"txn-participant-registration.run_cpu_time_us":303,"txn-participant-registration.run_wall_time_us":1982,"child_traces":[["op",{"apply.queue_time_us":37,"apply.run_cpu_time_us":357,"apply.run_wall_time_us":359,"prepare.queue_time_us":32,"prepare.run_cpu_time_us":249,"prepare.run_wall_time_us":249,"raft.queue_time_us":40,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":84,"replication_time_us":518,"thread_start_us":89,"threads_started":1,"wal-append.queue_time_us":161,"wal-append.run_cpu_time_us":374,"wal-append.run_wall_time_us":1000750,"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":42,"thread_start_us":65,"threads_started":1,"txn-participant-registration.queue_time_us":147}]]}]]}
W20250111 09:02:50.340783 22744 batcher.cc:441] Aborted: Failed to write batch of 1 ops to tablet 7bd9036707de43dbb1f76c1f97a24b68 after 1 attempt(s): Failed to write to server: 44cd71e01cd641bea1fafcb1fb7633d6 (127.14.61.1:34487): Write op should be aborted since it tries to acquire the partition lock that is held by another transaction that has lower txn ID
W20250111 09:02:50.343930 22778 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations
W20250111 09:02:50.355530 22793 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39942: Illegal state: Failed to write to server: 44cd71e01cd641bea1fafcb1fb7633d6 (127.14.61.1:34487): transaction ID 1 not open: ABORTED
I20250111 09:02:50.357290 14580 tablet_server.cc:178] TabletServer@127.14.61.1:0 shutting down...
I20250111 09:02:50.360230 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250111 09:02:50.360422 14580 tablet_replica.cc:331] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6: stopping tablet replica
I20250111 09:02:50.360532 14580 raft_consensus.cc:2238] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:02:50.360630 14580 raft_consensus.cc:2267] T 37c4c9fea8fd4771a9c8f6b4872d0bf7 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:02:50.361124 14580 tablet_replica.cc:331] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6: stopping tablet replica
I20250111 09:02:50.361218 14580 raft_consensus.cc:2238] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:02:50.361320 14580 raft_consensus.cc:2267] T 7bd9036707de43dbb1f76c1f97a24b68 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:02:50.361737 14580 tablet_replica.cc:331] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6: stopping tablet replica
I20250111 09:02:50.361891 14580 raft_consensus.cc:2238] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:02:50.361987 14580 raft_consensus.cc:2267] T a0634746f88040d7990a2440b83c7575 P 44cd71e01cd641bea1fafcb1fb7633d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:02:50.374727 14580 tablet_server.cc:195] TabletServer@127.14.61.1:0 shutdown complete.
I20250111 09:02:50.376852 14580 master.cc:537] Master@127.14.61.62:34079 shutting down...
I20250111 09:02:50.379936 14580 raft_consensus.cc:2238] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:02:50.380045 14580 raft_consensus.cc:2267] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:02:50.380101 14580 tablet_replica.cc:331] T 00000000000000000000000000000000 P 92b01fd1eb4f4112aa2067e6764c66c2: stopping tablet replica
I20250111 09:02:50.392046 14580 master.cc:559] Master@127.14.61.62:34079 shutdown complete.
[ OK ] TxnOpDispatcherITest.BeginTxnLockAbort (3934 ms)
[ RUN ] TxnOpDispatcherITest.BeginTxnLockRetry
I20250111 09:02:50.396550 14580 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.61.62:43179
I20250111 09:02:50.396817 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:50.398947 22804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:50.399631 22806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:50.400117 22803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:50.400729 14580 server_base.cc:1034] running on GCE node
I20250111 09:02:50.400864 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:02:50.400914 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:02:50.400929 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586170400928 us; error 0 us; skew 500 ppm
I20250111 09:02:50.401813 14580 webserver.cc:458] Webserver started at http://127.14.61.62:38463/ using document root <none> and password file <none>
I20250111 09:02:50.401952 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:50.401997 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:50.402067 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:02:50.402449 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.BeginTxnLockRetry.1736586070622223-14580-0/minicluster-data/master-0-root/instance:
uuid: "d90a7ff47f2341bebf48a73375998a33"
format_stamp: "Formatted at 2025-01-11 09:02:50 on dist-test-slave-nvbt"
I20250111 09:02:50.403685 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250111 09:02:50.404502 22811 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:50.404704 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:02:50.404762 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.BeginTxnLockRetry.1736586070622223-14580-0/minicluster-data/master-0-root
uuid: "d90a7ff47f2341bebf48a73375998a33"
format_stamp: "Formatted at 2025-01-11 09:02:50 on dist-test-slave-nvbt"
I20250111 09:02:50.404822 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.BeginTxnLockRetry.1736586070622223-14580-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.BeginTxnLockRetry.1736586070622223-14580-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.BeginTxnLockRetry.1736586070622223-14580-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:50.418139 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:50.418390 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:50.422894 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:43179
I20250111 09:02:50.423576 22873 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:43179 every 8 connection(s)
I20250111 09:02:50.423890 22874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:50.425376 22874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33: Bootstrap starting.
I20250111 09:02:50.426046 22874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:50.426829 22874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33: No bootstrap required, opened a new log
I20250111 09:02:50.427150 22874 raft_consensus.cc:357] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d90a7ff47f2341bebf48a73375998a33" member_type: VOTER }
I20250111 09:02:50.427217 22874 raft_consensus.cc:383] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:50.427254 22874 raft_consensus.cc:738] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d90a7ff47f2341bebf48a73375998a33, State: Initialized, Role: FOLLOWER
I20250111 09:02:50.427369 22874 consensus_queue.cc:260] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d90a7ff47f2341bebf48a73375998a33" member_type: VOTER }
I20250111 09:02:50.427430 22874 raft_consensus.cc:397] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:02:50.427484 22874 raft_consensus.cc:491] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:02:50.427529 22874 raft_consensus.cc:3054] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:50.428033 22874 raft_consensus.cc:513] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d90a7ff47f2341bebf48a73375998a33" member_type: VOTER }
I20250111 09:02:50.428129 22874 leader_election.cc:304] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d90a7ff47f2341bebf48a73375998a33; no voters:
I20250111 09:02:50.428268 22874 leader_election.cc:290] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:02:50.428366 22877 raft_consensus.cc:2798] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:50.428527 22877 raft_consensus.cc:695] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33 [term 1 LEADER]: Becoming Leader. State: Replica: d90a7ff47f2341bebf48a73375998a33, State: Running, Role: LEADER
I20250111 09:02:50.428633 22874 sys_catalog.cc:564] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33 [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:02:50.428640 22877 consensus_queue.cc:237] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d90a7ff47f2341bebf48a73375998a33" member_type: VOTER }
I20250111 09:02:50.429041 22878 sys_catalog.cc:455] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d90a7ff47f2341bebf48a73375998a33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d90a7ff47f2341bebf48a73375998a33" member_type: VOTER } }
I20250111 09:02:50.429054 22879 sys_catalog.cc:455] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d90a7ff47f2341bebf48a73375998a33. Latest consensus state: current_term: 1 leader_uuid: "d90a7ff47f2341bebf48a73375998a33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d90a7ff47f2341bebf48a73375998a33" member_type: VOTER } }
I20250111 09:02:50.429183 22879 sys_catalog.cc:458] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33 [sys.catalog]: This master's current role is: LEADER
I20250111 09:02:50.429387 22878 sys_catalog.cc:458] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33 [sys.catalog]: This master's current role is: LEADER
I20250111 09:02:50.429576 22885 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:02:50.430122 22885 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:02:50.430323 14580 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250111 09:02:50.431875 22885 catalog_manager.cc:1348] Generated new cluster ID: 159c60cd734b41f5a2528a2eb36b33b0
I20250111 09:02:50.431926 22885 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:02:50.442097 22885 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:02:50.442536 22885 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:02:50.449177 22885 catalog_manager.cc:5899] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33: Generated new TSK 0
I20250111 09:02:50.449303 22885 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:02:50.455687 22828 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47128:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:02:50.462325 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:02:50.464094 22901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:50.464670 22902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:02:50.465278 22904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:02:50.465905 14580 server_base.cc:1034] running on GCE node
I20250111 09:02:50.466054 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:02:50.466092 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:02:50.466107 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586170466107 us; error 0 us; skew 500 ppm
I20250111 09:02:50.466801 14580 webserver.cc:458] Webserver started at http://127.14.61.1:34839/ using document root <none> and password file <none>
I20250111 09:02:50.466915 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:02:50.466949 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:02:50.467012 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:02:50.467345 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.BeginTxnLockRetry.1736586070622223-14580-0/minicluster-data/ts-0-root/instance:
uuid: "bc4014674a6541d3a0586899f93f151f"
format_stamp: "Formatted at 2025-01-11 09:02:50 on dist-test-slave-nvbt"
I20250111 09:02:50.468559 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250111 09:02:50.469357 22909 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:50.469554 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:02:50.469606 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.BeginTxnLockRetry.1736586070622223-14580-0/minicluster-data/ts-0-root
uuid: "bc4014674a6541d3a0586899f93f151f"
format_stamp: "Formatted at 2025-01-11 09:02:50 on dist-test-slave-nvbt"
I20250111 09:02:50.469676 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.BeginTxnLockRetry.1736586070622223-14580-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.BeginTxnLockRetry.1736586070622223-14580-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.BeginTxnLockRetry.1736586070622223-14580-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:02:50.472841 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:02:50.473073 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:02:50.473630 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:02:50.473712 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:50.473794 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:02:50.473832 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:02:50.478436 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:46277
I20250111 09:02:50.478461 22979 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:46277 every 8 connection(s)
I20250111 09:02:50.479513 22980 heartbeater.cc:346] Connected to a master server at 127.14.61.62:43179
I20250111 09:02:50.479578 22980 heartbeater.cc:463] Registering TS with master...
I20250111 09:02:50.479751 22980 heartbeater.cc:510] Master 127.14.61.62:43179 requested a full tablet report, sending...
I20250111 09:02:50.480526 22828 ts_manager.cc:194] Registered new tserver with Master: bc4014674a6541d3a0586899f93f151f (127.14.61.1:46277)
I20250111 09:02:50.481011 14580 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002262572s
I20250111 09:02:50.481266 22828 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47140
I20250111 09:02:50.486581 22827 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47144:
name: "txn_op_dispatcher_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
I20250111 09:02:50.493487 22935 tablet_service.cc:1467] Processing CreateTablet for tablet d514891271754ef2809edf4ca19269c7 (DEFAULT_TABLE table=txn_op_dispatcher_test [id=d80e4842ac9a41cea4a3956bb74e8540]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:50.493556 22939 tablet_service.cc:1467] Processing CreateTablet for tablet c517c7e1a06242f48447448bd2038baa (DEFAULT_TABLE table=txn_op_dispatcher_test [id=d80e4842ac9a41cea4a3956bb74e8540]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:02:50.493714 22935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d514891271754ef2809edf4ca19269c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:50.493862 22939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c517c7e1a06242f48447448bd2038baa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:50.495380 22990 tablet_bootstrap.cc:492] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f: Bootstrap starting.
I20250111 09:02:50.496057 22990 tablet_bootstrap.cc:654] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:50.496831 22990 tablet_bootstrap.cc:492] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f: No bootstrap required, opened a new log
I20250111 09:02:50.496905 22990 ts_tablet_manager.cc:1397] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:02:50.497257 22990 raft_consensus.cc:357] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4014674a6541d3a0586899f93f151f" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 46277 } }
I20250111 09:02:50.497330 22990 raft_consensus.cc:383] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:50.497361 22990 raft_consensus.cc:738] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc4014674a6541d3a0586899f93f151f, State: Initialized, Role: FOLLOWER
I20250111 09:02:50.497474 22990 consensus_queue.cc:260] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4014674a6541d3a0586899f93f151f" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 46277 } }
I20250111 09:02:50.497543 22990 raft_consensus.cc:397] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:02:50.497584 22990 raft_consensus.cc:491] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:02:50.497629 22990 raft_consensus.cc:3054] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:50.498168 22990 raft_consensus.cc:513] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4014674a6541d3a0586899f93f151f" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 46277 } }
I20250111 09:02:50.498283 22990 leader_election.cc:304] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc4014674a6541d3a0586899f93f151f; no voters:
I20250111 09:02:50.498426 22990 leader_election.cc:290] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:02:50.498497 22992 raft_consensus.cc:2798] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:50.498682 22990 ts_tablet_manager.cc:1428] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250111 09:02:50.498680 22992 raft_consensus.cc:695] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f [term 1 LEADER]: Becoming Leader. State: Replica: bc4014674a6541d3a0586899f93f151f, State: Running, Role: LEADER
I20250111 09:02:50.498708 22980 heartbeater.cc:502] Master 127.14.61.62:43179 was elected leader, sending a full tablet report...
I20250111 09:02:50.498980 22990 tablet_bootstrap.cc:492] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f: Bootstrap starting.
I20250111 09:02:50.499151 22992 consensus_queue.cc:237] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4014674a6541d3a0586899f93f151f" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 46277 } }
I20250111 09:02:50.499758 22990 tablet_bootstrap.cc:654] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:50.500300 22827 catalog_manager.cc:5526] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f reported cstate change: term changed from 0 to 1, leader changed from <none> to bc4014674a6541d3a0586899f93f151f (127.14.61.1). New cstate: current_term: 1 leader_uuid: "bc4014674a6541d3a0586899f93f151f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4014674a6541d3a0586899f93f151f" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 46277 } health_report { overall_health: HEALTHY } } }
I20250111 09:02:50.500659 22990 tablet_bootstrap.cc:492] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f: No bootstrap required, opened a new log
I20250111 09:02:50.500727 22990 ts_tablet_manager.cc:1397] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20250111 09:02:50.501133 22990 raft_consensus.cc:357] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4014674a6541d3a0586899f93f151f" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 46277 } }
I20250111 09:02:50.501199 22990 raft_consensus.cc:383] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:50.501219 22990 raft_consensus.cc:738] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc4014674a6541d3a0586899f93f151f, State: Initialized, Role: FOLLOWER
I20250111 09:02:50.501324 22990 consensus_queue.cc:260] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4014674a6541d3a0586899f93f151f" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 46277 } }
I20250111 09:02:50.501387 22990 raft_consensus.cc:397] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:02:50.501410 22990 raft_consensus.cc:491] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:02:50.501438 22990 raft_consensus.cc:3054] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:50.502074 22990 raft_consensus.cc:513] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4014674a6541d3a0586899f93f151f" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 46277 } }
I20250111 09:02:50.502161 22990 leader_election.cc:304] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc4014674a6541d3a0586899f93f151f; no voters:
I20250111 09:02:50.502265 22990 leader_election.cc:290] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:02:50.502317 22992 raft_consensus.cc:2798] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:50.502396 22992 raft_consensus.cc:695] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f [term 1 LEADER]: Becoming Leader. State: Replica: bc4014674a6541d3a0586899f93f151f, State: Running, Role: LEADER
I20250111 09:02:50.502435 22990 ts_tablet_manager.cc:1428] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:02:50.502491 22992 consensus_queue.cc:237] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4014674a6541d3a0586899f93f151f" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 46277 } }
I20250111 09:02:50.503633 22827 catalog_manager.cc:5526] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f reported cstate change: term changed from 0 to 1, leader changed from <none> to bc4014674a6541d3a0586899f93f151f (127.14.61.1). New cstate: current_term: 1 leader_uuid: "bc4014674a6541d3a0586899f93f151f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4014674a6541d3a0586899f93f151f" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 46277 } health_report { overall_health: HEALTHY } } }
W20250111 09:02:50.509193 22986 txn_manager_proxy_rpc.cc:150] re-attempting BeginTransaction request to one of TxnManagers
I20250111 09:02:51.469866 22827 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47148:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:02:51.474300 22935 tablet_service.cc:1467] Processing CreateTablet for tablet 1eee1189847e455389f1141e96112bfd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cb02d9d864b04a70bfd2fb9c5dc05a80]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:02:51.474529 22935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1eee1189847e455389f1141e96112bfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:02:51.476277 23008 tablet_bootstrap.cc:492] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f: Bootstrap starting.
I20250111 09:02:51.476977 23008 tablet_bootstrap.cc:654] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f: Neither blocks nor log segments found. Creating new log.
I20250111 09:02:51.477896 23008 tablet_bootstrap.cc:492] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f: No bootstrap required, opened a new log
I20250111 09:02:51.477964 23008 ts_tablet_manager.cc:1397] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250111 09:02:51.478282 23008 raft_consensus.cc:357] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4014674a6541d3a0586899f93f151f" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 46277 } }
I20250111 09:02:51.478353 23008 raft_consensus.cc:383] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:02:51.478385 23008 raft_consensus.cc:738] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc4014674a6541d3a0586899f93f151f, State: Initialized, Role: FOLLOWER
I20250111 09:02:51.478492 23008 consensus_queue.cc:260] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4014674a6541d3a0586899f93f151f" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 46277 } }
I20250111 09:02:51.478550 23008 raft_consensus.cc:397] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:02:51.478582 23008 raft_consensus.cc:491] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:02:51.478628 23008 raft_consensus.cc:3054] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:02:51.479297 23008 raft_consensus.cc:513] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4014674a6541d3a0586899f93f151f" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 46277 } }
I20250111 09:02:51.479418 23008 leader_election.cc:304] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc4014674a6541d3a0586899f93f151f; no voters:
I20250111 09:02:51.479597 23008 leader_election.cc:290] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:02:51.479683 23010 raft_consensus.cc:2798] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:02:51.479871 23010 raft_consensus.cc:695] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f [term 1 LEADER]: Becoming Leader. State: Replica: bc4014674a6541d3a0586899f93f151f, State: Running, Role: LEADER
I20250111 09:02:51.479905 23008 ts_tablet_manager.cc:1428] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20250111 09:02:51.479984 22981 tablet.cc:2367] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250111 09:02:51.480022 23010 consensus_queue.cc:237] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4014674a6541d3a0586899f93f151f" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 46277 } }
I20250111 09:02:51.480418 23011 tablet_replica.cc:440] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc4014674a6541d3a0586899f93f151f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4014674a6541d3a0586899f93f151f" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 46277 } } }
I20250111 09:02:51.480502 23011 tablet_replica.cc:443] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:02:51.480923 23014 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:02:51.480892 23012 tablet_replica.cc:440] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f: TxnStatusTablet state changed. Reason: New leader bc4014674a6541d3a0586899f93f151f. Latest consensus state: current_term: 1 leader_uuid: "bc4014674a6541d3a0586899f93f151f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4014674a6541d3a0586899f93f151f" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 46277 } } }
I20250111 09:02:51.481130 23012 tablet_replica.cc:443] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:02:51.481316 23014 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:02:51.481261 22827 catalog_manager.cc:5526] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f reported cstate change: term changed from 0 to 1, leader changed from <none> to bc4014674a6541d3a0586899f93f151f (127.14.61.1). New cstate: current_term: 1 leader_uuid: "bc4014674a6541d3a0586899f93f151f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4014674a6541d3a0586899f93f151f" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 46277 } health_report { overall_health: HEALTHY } } }
W20250111 09:02:53.196550 23019 lock_manager.cc:494] Txn 0 has not acquired the partition lock after 1000ms
W20250111 09:02:53.196678 23019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:36566: Service unavailable: Write op should retry since it tries to acquire the partition lock that is held by another transaction that has higher txn ID
I20250111 09:02:53.196815 23019 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:36566 (ReqId={client: 82da8ff2475d4b719c38054dd88bf6d6, seq_no=5, attempt_no=0}) took 1003 ms. Trace:
I20250111 09:02:53.196844 23019 rpcz_store.cc:276] 0111 09:02:52.193487 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:52.193537 (+ 50us) service_pool.cc:225] Handling call
0111 09:02:53.196806 (+1003269us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:52.195393 (+ 0us) participant_op.cc:163] PREPARE: starting
0111 09:02:52.195412 (+ 19us) participant_op.cc:199] PREPARE: finished
0111 09:02:52.195423 (+ 11us) participant_op.cc:212] START. Timestamp: 1736586172195417
0111 09:02:52.195424 (+ 1us) op_driver.cc:348] REPLICATION: starting
0111 09:02:52.195480 (+ 56us) log.cc:844] Serialized 65 byte log entry
0111 09:02:52.195725 (+ 245us) op_driver.cc:464] REPLICATION: finished
0111 09:02:52.195791 (+ 66us) participant_op.cc:265] APPLY: starting
0111 09:02:52.195895 (+ 104us) participant_op.cc:270] APPLY: finished
0111 09:02:52.195912 (+ 17us) log.cc:844] Serialized 14 byte log entry
0111 09:02:52.195935 (+ 23us) participant_op.cc:94] Released txn lock
0111 09:02:52.195939 (+ 4us) participant_op.cc:291] FINISH: Op applied
Related trace 'op':
0111 09:02:52.196079 (+ 0us) write_op.cc:183] PREPARE: starting on tablet c517c7e1a06242f48447448bd2038baa
0111 09:02:52.196126 (+ 47us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:52.196127 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:02:52.196128 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:52.196155 (+ 27us) write_op.cc:620] Acquiring the partition lock for write op
0111 09:02:53.196656 (+1000501us) op_driver.cc:389] HandleFailure(Service unavailable: Write op should retry since it tries to acquire the partition lock that is held by another transaction that has higher txn ID)
0111 09:02:53.196666 (+ 10us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:53.196671 (+ 5us) write_op.cc:454] Released schema lock
0111 09:02:53.196676 (+ 5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"txn-participant-registration.queue_time_us":34,"txn-participant-registration.run_cpu_time_us":273,"txn-participant-registration.run_wall_time_us":1660,"child_traces":[["op",{"apply.queue_time_us":32,"apply.run_cpu_time_us":284,"apply.run_wall_time_us":285,"prepare.queue_time_us":38,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":118,"replication_time_us":286,"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":33}]]}]]}
W20250111 09:02:54.201797 23019 lock_manager.cc:494] Txn 0 has not acquired the partition lock after 1000ms
W20250111 09:02:54.201967 23019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:36566: Service unavailable: Write op should retry since it tries to acquire the partition lock that is held by another transaction that has higher txn ID
I20250111 09:02:54.202129 23019 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:36566 (ReqId={client: 82da8ff2475d4b719c38054dd88bf6d6, seq_no=5, attempt_no=1}) took 1001 ms. Trace:
I20250111 09:02:54.202165 23019 rpcz_store.cc:276] 0111 09:02:53.200744 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:53.200804 (+ 60us) service_pool.cc:225] Handling call
0111 09:02:54.202119 (+1001315us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:53.201198 (+ 0us) write_op.cc:183] PREPARE: starting on tablet c517c7e1a06242f48447448bd2038baa
0111 09:02:53.201310 (+ 112us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:53.201311 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:02:53.201312 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:53.201367 (+ 55us) write_op.cc:620] Acquiring the partition lock for write op
0111 09:02:54.201937 (+1000570us) op_driver.cc:389] HandleFailure(Service unavailable: Write op should retry since it tries to acquire the partition lock that is held by another transaction that has higher txn ID)
0111 09:02:54.201950 (+ 13us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:54.201956 (+ 6us) write_op.cc:454] Released schema lock
0111 09:02:54.201962 (+ 6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":32}]]}
W20250111 09:02:55.206218 23019 lock_manager.cc:494] Txn 0 has not acquired the partition lock after 1000ms
W20250111 09:02:55.206390 23019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:36566: Service unavailable: Write op should retry since it tries to acquire the partition lock that is held by another transaction that has higher txn ID
I20250111 09:02:55.206542 23019 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:36566 (ReqId={client: 82da8ff2475d4b719c38054dd88bf6d6, seq_no=5, attempt_no=2}) took 1001 ms. Trace:
I20250111 09:02:55.206573 23019 rpcz_store.cc:276] 0111 09:02:54.205118 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:54.205198 (+ 80us) service_pool.cc:225] Handling call
0111 09:02:55.206531 (+1001333us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:54.205622 (+ 0us) write_op.cc:183] PREPARE: starting on tablet c517c7e1a06242f48447448bd2038baa
0111 09:02:54.205769 (+ 147us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:54.205770 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:02:54.205771 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:54.205830 (+ 59us) write_op.cc:620] Acquiring the partition lock for write op
0111 09:02:55.206364 (+1000534us) op_driver.cc:389] HandleFailure(Service unavailable: Write op should retry since it tries to acquire the partition lock that is held by another transaction that has higher txn ID)
0111 09:02:55.206377 (+ 13us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:55.206381 (+ 4us) write_op.cc:454] Released schema lock
0111 09:02:55.206387 (+ 6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":55}]]}
W20250111 09:02:56.211560 23019 lock_manager.cc:494] Txn 0 has not acquired the partition lock after 1000ms
W20250111 09:02:56.211709 23019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:36566: Service unavailable: Write op should retry since it tries to acquire the partition lock that is held by another transaction that has higher txn ID
I20250111 09:02:56.211866 23019 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:36566 (ReqId={client: 82da8ff2475d4b719c38054dd88bf6d6, seq_no=5, attempt_no=3}) took 1001 ms. Trace:
I20250111 09:02:56.211901 23019 rpcz_store.cc:276] 0111 09:02:55.210521 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:55.210581 (+ 60us) service_pool.cc:225] Handling call
0111 09:02:56.211856 (+1001275us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:55.210975 (+ 0us) write_op.cc:183] PREPARE: starting on tablet c517c7e1a06242f48447448bd2038baa
0111 09:02:55.211070 (+ 95us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:55.211072 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:02:55.211073 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:55.211127 (+ 54us) write_op.cc:620] Acquiring the partition lock for write op
0111 09:02:56.211680 (+1000553us) op_driver.cc:389] HandleFailure(Service unavailable: Write op should retry since it tries to acquire the partition lock that is held by another transaction that has higher txn ID)
0111 09:02:56.211693 (+ 13us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:56.211699 (+ 6us) write_op.cc:454] Released schema lock
0111 09:02:56.211704 (+ 5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":54}]]}
W20250111 09:02:57.219991 23019 lock_manager.cc:494] Txn 0 has not acquired the partition lock after 1000ms
W20250111 09:02:57.220152 23019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:36566: Service unavailable: Write op should retry since it tries to acquire the partition lock that is held by another transaction that has higher txn ID
I20250111 09:02:57.220306 23019 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:36566 (ReqId={client: 82da8ff2475d4b719c38054dd88bf6d6, seq_no=5, attempt_no=4}) took 1001 ms. Trace:
I20250111 09:02:57.220343 23019 rpcz_store.cc:276] 0111 09:02:56.218892 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:56.219010 (+ 118us) service_pool.cc:225] Handling call
0111 09:02:57.220296 (+1001286us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:56.219411 (+ 0us) write_op.cc:183] PREPARE: starting on tablet c517c7e1a06242f48447448bd2038baa
0111 09:02:56.219504 (+ 93us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:56.219505 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:02:56.219506 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:56.219558 (+ 52us) write_op.cc:620] Acquiring the partition lock for write op
0111 09:02:57.220124 (+1000566us) op_driver.cc:389] HandleFailure(Service unavailable: Write op should retry since it tries to acquire the partition lock that is held by another transaction that has higher txn ID)
0111 09:02:57.220137 (+ 13us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:57.220142 (+ 5us) write_op.cc:454] Released schema lock
0111 09:02:57.220148 (+ 6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":52}]]}
W20250111 09:02:58.229341 23019 lock_manager.cc:494] Txn 0 has not acquired the partition lock after 1000ms
W20250111 09:02:58.229485 23019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:36566: Service unavailable: Write op should retry since it tries to acquire the partition lock that is held by another transaction that has higher txn ID
I20250111 09:02:58.229631 23019 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:36566 (ReqId={client: 82da8ff2475d4b719c38054dd88bf6d6, seq_no=5, attempt_no=5}) took 1001 ms. Trace:
I20250111 09:02:58.229704 23019 rpcz_store.cc:276] 0111 09:02:57.228340 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:57.228401 (+ 61us) service_pool.cc:225] Handling call
0111 09:02:58.229621 (+1001220us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:57.228820 (+ 0us) write_op.cc:183] PREPARE: starting on tablet c517c7e1a06242f48447448bd2038baa
0111 09:02:57.228913 (+ 93us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:57.228914 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:02:57.228915 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:57.228969 (+ 54us) write_op.cc:620] Acquiring the partition lock for write op
0111 09:02:58.229457 (+1000488us) op_driver.cc:389] HandleFailure(Service unavailable: Write op should retry since it tries to acquire the partition lock that is held by another transaction that has higher txn ID)
0111 09:02:58.229470 (+ 13us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:58.229476 (+ 6us) write_op.cc:454] Released schema lock
0111 09:02:58.229481 (+ 5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":44}]]}
W20250111 09:02:59.238735 23019 lock_manager.cc:494] Txn 0 has not acquired the partition lock after 1000ms
W20250111 09:02:59.238899 23019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:36566: Service unavailable: Write op should retry since it tries to acquire the partition lock that is held by another transaction that has higher txn ID
I20250111 09:02:59.239099 23019 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:36566 (ReqId={client: 82da8ff2475d4b719c38054dd88bf6d6, seq_no=5, attempt_no=6}) took 1001 ms. Trace:
I20250111 09:02:59.239158 23019 rpcz_store.cc:276] 0111 09:02:58.237653 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:58.237738 (+ 85us) service_pool.cc:225] Handling call
0111 09:02:59.239083 (+1001345us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:58.238180 (+ 0us) write_op.cc:183] PREPARE: starting on tablet c517c7e1a06242f48447448bd2038baa
0111 09:02:58.238287 (+ 107us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:58.238288 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:02:58.238289 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:58.238342 (+ 53us) write_op.cc:620] Acquiring the partition lock for write op
0111 09:02:59.238864 (+1000522us) op_driver.cc:389] HandleFailure(Service unavailable: Write op should retry since it tries to acquire the partition lock that is held by another transaction that has higher txn ID)
0111 09:02:59.238878 (+ 14us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:02:59.238885 (+ 7us) write_op.cc:454] Released schema lock
0111 09:02:59.238893 (+ 8us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":47}]]}
W20250111 09:03:00.252195 23019 lock_manager.cc:494] Txn 0 has not acquired the partition lock after 1000ms
W20250111 09:03:00.252358 23019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:36566: Service unavailable: Write op should retry since it tries to acquire the partition lock that is held by another transaction that has higher txn ID
I20250111 09:03:00.252516 23019 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:36566 (ReqId={client: 82da8ff2475d4b719c38054dd88bf6d6, seq_no=5, attempt_no=7}) took 1001 ms. Trace:
I20250111 09:03:00.252554 23019 rpcz_store.cc:276] 0111 09:02:59.251152 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:02:59.251206 (+ 54us) service_pool.cc:225] Handling call
0111 09:03:00.252505 (+1001299us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:02:59.251589 (+ 0us) write_op.cc:183] PREPARE: starting on tablet c517c7e1a06242f48447448bd2038baa
0111 09:02:59.251685 (+ 96us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:02:59.251686 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:02:59.251687 (+ 1us) tablet.cc:659] Decoding operations
0111 09:02:59.251763 (+ 76us) write_op.cc:620] Acquiring the partition lock for write op
0111 09:03:00.252332 (+1000569us) op_driver.cc:389] HandleFailure(Service unavailable: Write op should retry since it tries to acquire the partition lock that is held by another transaction that has higher txn ID)
0111 09:03:00.252344 (+ 12us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:03:00.252349 (+ 5us) write_op.cc:454] Released schema lock
0111 09:03:00.252354 (+ 5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":51}]]}
W20250111 09:03:01.262542 23019 lock_manager.cc:494] Txn 0 has not acquired the partition lock after 1000ms
W20250111 09:03:01.262699 23019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:36566: Service unavailable: Write op should retry since it tries to acquire the partition lock that is held by another transaction that has higher txn ID
I20250111 09:03:01.262868 23019 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:36566 (ReqId={client: 82da8ff2475d4b719c38054dd88bf6d6, seq_no=5, attempt_no=8}) took 1001 ms. Trace:
I20250111 09:03:01.262909 23019 rpcz_store.cc:276] 0111 09:03:00.261419 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:00.261477 (+ 58us) service_pool.cc:225] Handling call
0111 09:03:01.262857 (+1001380us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:03:00.261952 (+ 0us) write_op.cc:183] PREPARE: starting on tablet c517c7e1a06242f48447448bd2038baa
0111 09:03:00.262056 (+ 104us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:03:00.262057 (+ 1us) write_op.cc:435] Acquired schema lock
0111 09:03:00.262058 (+ 1us) tablet.cc:659] Decoding operations
0111 09:03:00.262114 (+ 56us) write_op.cc:620] Acquiring the partition lock for write op
0111 09:03:01.262672 (+1000558us) op_driver.cc:389] HandleFailure(Service unavailable: Write op should retry since it tries to acquire the partition lock that is held by another transaction that has higher txn ID)
0111 09:03:01.262684 (+ 12us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:03:01.262689 (+ 5us) write_op.cc:454] Released schema lock
0111 09:03:01.262695 (+ 6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":156}]]}
W20250111 09:03:02.277784 23019 lock_manager.cc:494] Txn 0 has not acquired the partition lock after 1000ms
W20250111 09:03:02.277947 23019 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:36566: Service unavailable: Write op should retry since it tries to acquire the partition lock that is held by another transaction that has higher txn ID
I20250111 09:03:02.278139 23019 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:36566 (ReqId={client: 82da8ff2475d4b719c38054dd88bf6d6, seq_no=5, attempt_no=9}) took 1001 ms. Trace:
I20250111 09:03:02.278191 23019 rpcz_store.cc:276] 0111 09:03:01.276779 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:01.276830 (+ 51us) service_pool.cc:225] Handling call
0111 09:03:02.278125 (+1001295us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0111 09:03:01.277237 (+ 0us) write_op.cc:183] PREPARE: starting on tablet c517c7e1a06242f48447448bd2038baa
0111 09:03:01.277335 (+ 98us) write_op.cc:432] Acquiring schema lock in shared mode
0111 09:03:01.277337 (+ 2us) write_op.cc:435] Acquired schema lock
0111 09:03:01.277338 (+ 1us) tablet.cc:659] Decoding operations
0111 09:03:01.277390 (+ 52us) write_op.cc:620] Acquiring the partition lock for write op
0111 09:03:02.277917 (+1000527us) op_driver.cc:389] HandleFailure(Service unavailable: Write op should retry since it tries to acquire the partition lock that is held by another transaction that has higher txn ID)
0111 09:03:02.277931 (+ 14us) write_op.cc:489] Releasing partition, row and schema locks
0111 09:03:02.277937 (+ 6us) write_op.cc:454] Released schema lock
0111 09:03:02.277943 (+ 6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":35}]]}
W20250111 09:03:02.290920 22986 batcher.cc:441] Timed out: Failed to write batch of 1 ops to tablet c517c7e1a06242f48447448bd2038baa after 11 attempt(s): Failed to write to server: (no server available): Write(tablet: c517c7e1a06242f48447448bd2038baa, num_ops: 1, num_attempts: 11) passed its deadline
I20250111 09:03:02.302886 14580 tablet_server.cc:178] TabletServer@127.14.61.1:0 shutting down...
I20250111 09:03:02.306138 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250111 09:03:02.306303 14580 tablet_replica.cc:331] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f: stopping tablet replica
I20250111 09:03:02.306430 14580 raft_consensus.cc:2238] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:02.306532 14580 raft_consensus.cc:2267] T 1eee1189847e455389f1141e96112bfd P bc4014674a6541d3a0586899f93f151f [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:02.306962 14580 tablet_replica.cc:331] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f: stopping tablet replica
I20250111 09:03:02.307040 14580 raft_consensus.cc:2238] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:02.307117 14580 raft_consensus.cc:2267] T d514891271754ef2809edf4ca19269c7 P bc4014674a6541d3a0586899f93f151f [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:02.307499 14580 tablet_replica.cc:331] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f: stopping tablet replica
I20250111 09:03:02.307567 14580 raft_consensus.cc:2238] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:02.307641 14580 raft_consensus.cc:2267] T c517c7e1a06242f48447448bd2038baa P bc4014674a6541d3a0586899f93f151f [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:02.320622 14580 tablet_server.cc:195] TabletServer@127.14.61.1:0 shutdown complete.
I20250111 09:03:02.322835 14580 master.cc:537] Master@127.14.61.62:43179 shutting down...
I20250111 09:03:02.326380 14580 raft_consensus.cc:2238] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:02.326506 14580 raft_consensus.cc:2267] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:02.326575 14580 tablet_replica.cc:331] T 00000000000000000000000000000000 P d90a7ff47f2341bebf48a73375998a33: stopping tablet replica
I20250111 09:03:02.338464 14580 master.cc:559] Master@127.14.61.62:43179 shutdown complete.
[ OK ] TxnOpDispatcherITest.BeginTxnLockRetry (11946 ms)
[ RUN ] TxnOpDispatcherITest.ErrorInParticipantRegistration
I20250111 09:03:02.343376 14580 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.61.62:42337
I20250111 09:03:02.343683 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:02.345937 23037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:02.346735 14580 server_base.cc:1034] running on GCE node
W20250111 09:03:02.347414 23040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:02.348011 23038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:02.348181 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:02.348220 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:02.348238 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586182348238 us; error 0 us; skew 500 ppm
I20250111 09:03:02.349050 14580 webserver.cc:458] Webserver started at http://127.14.61.62:45175/ using document root <none> and password file <none>
I20250111 09:03:02.349180 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:02.349217 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:02.349267 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:02.349594 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.ErrorInParticipantRegistration.1736586070622223-14580-0/minicluster-data/master-0-root/instance:
uuid: "a08a45538bd341cbb4842219b288b64e"
format_stamp: "Formatted at 2025-01-11 09:03:02 on dist-test-slave-nvbt"
I20250111 09:03:02.350921 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250111 09:03:02.351728 23045 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:02.351969 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20250111 09:03:02.352036 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.ErrorInParticipantRegistration.1736586070622223-14580-0/minicluster-data/master-0-root
uuid: "a08a45538bd341cbb4842219b288b64e"
format_stamp: "Formatted at 2025-01-11 09:03:02 on dist-test-slave-nvbt"
I20250111 09:03:02.352103 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.ErrorInParticipantRegistration.1736586070622223-14580-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.ErrorInParticipantRegistration.1736586070622223-14580-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.ErrorInParticipantRegistration.1736586070622223-14580-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:02.366170 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:02.366446 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:02.370724 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:42337
I20250111 09:03:02.372260 23107 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:42337 every 8 connection(s)
I20250111 09:03:02.372645 23108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:02.374261 23108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e: Bootstrap starting.
I20250111 09:03:02.374972 23108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:02.375804 23108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e: No bootstrap required, opened a new log
I20250111 09:03:02.376137 23108 raft_consensus.cc:357] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08a45538bd341cbb4842219b288b64e" member_type: VOTER }
I20250111 09:03:02.376233 23108 raft_consensus.cc:383] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:02.376284 23108 raft_consensus.cc:738] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a08a45538bd341cbb4842219b288b64e, State: Initialized, Role: FOLLOWER
I20250111 09:03:02.376413 23108 consensus_queue.cc:260] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08a45538bd341cbb4842219b288b64e" member_type: VOTER }
I20250111 09:03:02.376482 23108 raft_consensus.cc:397] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:02.376520 23108 raft_consensus.cc:491] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:02.376574 23108 raft_consensus.cc:3054] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:02.377205 23108 raft_consensus.cc:513] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08a45538bd341cbb4842219b288b64e" member_type: VOTER }
I20250111 09:03:02.377327 23108 leader_election.cc:304] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a08a45538bd341cbb4842219b288b64e; no voters:
I20250111 09:03:02.377486 23108 leader_election.cc:290] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:02.377604 23111 raft_consensus.cc:2798] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:02.377832 23111 raft_consensus.cc:695] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e [term 1 LEADER]: Becoming Leader. State: Replica: a08a45538bd341cbb4842219b288b64e, State: Running, Role: LEADER
I20250111 09:03:02.377883 23108 sys_catalog.cc:564] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:03:02.377964 23111 consensus_queue.cc:237] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08a45538bd341cbb4842219b288b64e" member_type: VOTER }
I20250111 09:03:02.378386 23112 sys_catalog.cc:455] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a08a45538bd341cbb4842219b288b64e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08a45538bd341cbb4842219b288b64e" member_type: VOTER } }
I20250111 09:03:02.378532 23112 sys_catalog.cc:458] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:02.378407 23113 sys_catalog.cc:455] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a08a45538bd341cbb4842219b288b64e. Latest consensus state: current_term: 1 leader_uuid: "a08a45538bd341cbb4842219b288b64e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08a45538bd341cbb4842219b288b64e" member_type: VOTER } }
I20250111 09:03:02.378697 23113 sys_catalog.cc:458] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:02.379285 23118 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:03:02.379892 23118 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:03:02.380085 14580 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250111 09:03:02.381793 23118 catalog_manager.cc:1348] Generated new cluster ID: 07cb10f1e65e449996fce33046d320e1
I20250111 09:03:02.381847 23118 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:03:02.390605 23118 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:03:02.391067 23118 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:03:02.396389 23118 catalog_manager.cc:5899] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e: Generated new TSK 0
I20250111 09:03:02.396529 23118 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:03:02.398998 23062 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53606:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:02.412194 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:02.414511 23136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:02.415091 23135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:02.415885 23138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:02.416543 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:02.417306 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:02.417363 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:02.417384 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586182417384 us; error 0 us; skew 500 ppm
I20250111 09:03:02.418655 14580 webserver.cc:458] Webserver started at http://127.14.61.1:33519/ using document root <none> and password file <none>
I20250111 09:03:02.418826 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:02.418884 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:02.418962 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:02.419415 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.ErrorInParticipantRegistration.1736586070622223-14580-0/minicluster-data/ts-0-root/instance:
uuid: "6924a6cc6c904e24bb17911b10b321df"
format_stamp: "Formatted at 2025-01-11 09:03:02 on dist-test-slave-nvbt"
I20250111 09:03:02.421319 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20250111 09:03:02.422345 23143 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:02.422623 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:02.422722 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.ErrorInParticipantRegistration.1736586070622223-14580-0/minicluster-data/ts-0-root
uuid: "6924a6cc6c904e24bb17911b10b321df"
format_stamp: "Formatted at 2025-01-11 09:03:02 on dist-test-slave-nvbt"
I20250111 09:03:02.422796 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.ErrorInParticipantRegistration.1736586070622223-14580-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.ErrorInParticipantRegistration.1736586070622223-14580-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.ErrorInParticipantRegistration.1736586070622223-14580-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:02.437636 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:02.437919 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:02.438572 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:03:02.438616 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:02.438661 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:03:02.438699 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:02.444079 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:39327
I20250111 09:03:02.444110 23212 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:39327 every 8 connection(s)
I20250111 09:03:02.445089 23214 heartbeater.cc:346] Connected to a master server at 127.14.61.62:42337
I20250111 09:03:02.445173 23214 heartbeater.cc:463] Registering TS with master...
I20250111 09:03:02.445360 23214 heartbeater.cc:510] Master 127.14.61.62:42337 requested a full tablet report, sending...
I20250111 09:03:02.445904 23062 ts_manager.cc:194] Registered new tserver with Master: 6924a6cc6c904e24bb17911b10b321df (127.14.61.1:39327)
I20250111 09:03:02.446556 23062 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53612
I20250111 09:03:02.446877 14580 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002383791s
I20250111 09:03:02.453287 23062 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53630:
name: "txn_op_dispatcher_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
I20250111 09:03:02.460958 23173 tablet_service.cc:1467] Processing CreateTablet for tablet bbb8dc5b33d44453be186453374370f6 (DEFAULT_TABLE table=txn_op_dispatcher_test [id=8194b60c5a3d4ae38848b686c9e1de3c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:02.461035 23172 tablet_service.cc:1467] Processing CreateTablet for tablet 6226b69870b945678845fc9ea1f9c188 (DEFAULT_TABLE table=txn_op_dispatcher_test [id=8194b60c5a3d4ae38848b686c9e1de3c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:02.461247 23173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbb8dc5b33d44453be186453374370f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:02.461397 23172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6226b69870b945678845fc9ea1f9c188. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:02.463174 23224 tablet_bootstrap.cc:492] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df: Bootstrap starting.
I20250111 09:03:02.464061 23224 tablet_bootstrap.cc:654] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:02.464885 23224 tablet_bootstrap.cc:492] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df: No bootstrap required, opened a new log
I20250111 09:03:02.464951 23224 ts_tablet_manager.cc:1397] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:02.465345 23224 raft_consensus.cc:357] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6924a6cc6c904e24bb17911b10b321df" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39327 } }
I20250111 09:03:02.465428 23224 raft_consensus.cc:383] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:02.465451 23224 raft_consensus.cc:738] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6924a6cc6c904e24bb17911b10b321df, State: Initialized, Role: FOLLOWER
I20250111 09:03:02.465564 23224 consensus_queue.cc:260] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6924a6cc6c904e24bb17911b10b321df" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39327 } }
I20250111 09:03:02.465639 23224 raft_consensus.cc:397] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:02.465663 23224 raft_consensus.cc:491] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:02.465699 23224 raft_consensus.cc:3054] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:02.466306 23224 raft_consensus.cc:513] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6924a6cc6c904e24bb17911b10b321df" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39327 } }
I20250111 09:03:02.466419 23224 leader_election.cc:304] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6924a6cc6c904e24bb17911b10b321df; no voters:
I20250111 09:03:02.466602 23224 leader_election.cc:290] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:02.466684 23226 raft_consensus.cc:2798] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:02.466913 23224 ts_tablet_manager.cc:1428] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:02.466984 23226 raft_consensus.cc:695] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df [term 1 LEADER]: Becoming Leader. State: Replica: 6924a6cc6c904e24bb17911b10b321df, State: Running, Role: LEADER
I20250111 09:03:02.467118 23214 heartbeater.cc:502] Master 127.14.61.62:42337 was elected leader, sending a full tablet report...
I20250111 09:03:02.467136 23224 tablet_bootstrap.cc:492] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df: Bootstrap starting.
I20250111 09:03:02.467135 23226 consensus_queue.cc:237] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6924a6cc6c904e24bb17911b10b321df" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39327 } }
I20250111 09:03:02.468034 23224 tablet_bootstrap.cc:654] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:02.468364 23062 catalog_manager.cc:5526] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df reported cstate change: term changed from 0 to 1, leader changed from <none> to 6924a6cc6c904e24bb17911b10b321df (127.14.61.1). New cstate: current_term: 1 leader_uuid: "6924a6cc6c904e24bb17911b10b321df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6924a6cc6c904e24bb17911b10b321df" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39327 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:02.468989 23224 tablet_bootstrap.cc:492] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df: No bootstrap required, opened a new log
I20250111 09:03:02.469085 23224 ts_tablet_manager.cc:1397] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:02.469426 23224 raft_consensus.cc:357] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6924a6cc6c904e24bb17911b10b321df" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39327 } }
I20250111 09:03:02.469509 23224 raft_consensus.cc:383] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:02.469545 23224 raft_consensus.cc:738] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6924a6cc6c904e24bb17911b10b321df, State: Initialized, Role: FOLLOWER
I20250111 09:03:02.469648 23224 consensus_queue.cc:260] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6924a6cc6c904e24bb17911b10b321df" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39327 } }
I20250111 09:03:02.469725 23224 raft_consensus.cc:397] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:02.469769 23224 raft_consensus.cc:491] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:02.469815 23224 raft_consensus.cc:3054] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:02.470590 23224 raft_consensus.cc:513] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6924a6cc6c904e24bb17911b10b321df" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39327 } }
I20250111 09:03:02.470695 23224 leader_election.cc:304] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6924a6cc6c904e24bb17911b10b321df; no voters:
I20250111 09:03:02.470798 23224 leader_election.cc:290] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:02.470844 23227 raft_consensus.cc:2798] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:02.470978 23227 raft_consensus.cc:695] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df [term 1 LEADER]: Becoming Leader. State: Replica: 6924a6cc6c904e24bb17911b10b321df, State: Running, Role: LEADER
I20250111 09:03:02.471045 23224 ts_tablet_manager.cc:1428] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250111 09:03:02.471079 23227 consensus_queue.cc:237] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6924a6cc6c904e24bb17911b10b321df" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39327 } }
I20250111 09:03:02.472138 23062 catalog_manager.cc:5526] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df reported cstate change: term changed from 0 to 1, leader changed from <none> to 6924a6cc6c904e24bb17911b10b321df (127.14.61.1). New cstate: current_term: 1 leader_uuid: "6924a6cc6c904e24bb17911b10b321df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6924a6cc6c904e24bb17911b10b321df" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39327 } health_report { overall_health: HEALTHY } } }
W20250111 09:03:02.476482 23217 txn_manager_proxy_rpc.cc:150] re-attempting BeginTransaction request to one of TxnManagers
I20250111 09:03:03.406380 23062 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53644:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:03.410501 23173 tablet_service.cc:1467] Processing CreateTablet for tablet 3b3b19dae82440a2ab50e34eb5a7e9fa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a1c90234592247e5a4cfc4fa537a6881]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:03:03.410750 23173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b3b19dae82440a2ab50e34eb5a7e9fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:03.412556 23242 tablet_bootstrap.cc:492] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df: Bootstrap starting.
I20250111 09:03:03.413441 23242 tablet_bootstrap.cc:654] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:03.414216 23242 tablet_bootstrap.cc:492] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df: No bootstrap required, opened a new log
I20250111 09:03:03.414292 23242 ts_tablet_manager.cc:1397] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:03.414615 23242 raft_consensus.cc:357] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6924a6cc6c904e24bb17911b10b321df" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39327 } }
I20250111 09:03:03.414687 23242 raft_consensus.cc:383] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:03.414718 23242 raft_consensus.cc:738] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6924a6cc6c904e24bb17911b10b321df, State: Initialized, Role: FOLLOWER
I20250111 09:03:03.414830 23242 consensus_queue.cc:260] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6924a6cc6c904e24bb17911b10b321df" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39327 } }
I20250111 09:03:03.414898 23242 raft_consensus.cc:397] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:03.414943 23242 raft_consensus.cc:491] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:03.414988 23242 raft_consensus.cc:3054] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:03.415575 23242 raft_consensus.cc:513] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6924a6cc6c904e24bb17911b10b321df" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39327 } }
I20250111 09:03:03.415679 23242 leader_election.cc:304] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6924a6cc6c904e24bb17911b10b321df; no voters:
I20250111 09:03:03.415829 23242 leader_election.cc:290] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:03.415900 23244 raft_consensus.cc:2798] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:03.416045 23244 raft_consensus.cc:695] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df [term 1 LEADER]: Becoming Leader. State: Replica: 6924a6cc6c904e24bb17911b10b321df, State: Running, Role: LEADER
I20250111 09:03:03.416121 23242 ts_tablet_manager.cc:1428] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:03.416214 23244 consensus_queue.cc:237] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6924a6cc6c904e24bb17911b10b321df" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39327 } }
I20250111 09:03:03.416611 23245 tablet_replica.cc:440] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df: TxnStatusTablet state changed. Reason: New leader 6924a6cc6c904e24bb17911b10b321df. Latest consensus state: current_term: 1 leader_uuid: "6924a6cc6c904e24bb17911b10b321df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6924a6cc6c904e24bb17911b10b321df" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39327 } } }
I20250111 09:03:03.416715 23245 tablet_replica.cc:443] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:03.416635 23246 tablet_replica.cc:440] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6924a6cc6c904e24bb17911b10b321df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6924a6cc6c904e24bb17911b10b321df" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39327 } } }
I20250111 09:03:03.416945 23246 tablet_replica.cc:443] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:03.417138 23248 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:03:03.417208 23248 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:03:03.417309 23062 catalog_manager.cc:5526] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df reported cstate change: term changed from 0 to 1, leader changed from <none> to 6924a6cc6c904e24bb17911b10b321df (127.14.61.1). New cstate: current_term: 1 leader_uuid: "6924a6cc6c904e24bb17911b10b321df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6924a6cc6c904e24bb17911b10b321df" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39327 } health_report { overall_health: HEALTHY } } }
W20250111 09:03:03.474053 23255 tablet_replica.cc:1307] Invalid argument: Failed to write to server: 6924a6cc6c904e24bb17911b10b321df (127.14.61.1:39327): transaction ID 10 not found, current highest txn ID: 0: cancelling pending write operations [suppressed 1 similar messages]
W20250111 09:03:03.474141 23255 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40504: Invalid argument: Failed to write to server: 6924a6cc6c904e24bb17911b10b321df (127.14.61.1:39327): transaction ID 10 not found, current highest txn ID: 0
W20250111 09:03:03.474431 23218 batcher.cc:441] Invalid argument: Failed to write batch of 1 ops to tablet bbb8dc5b33d44453be186453374370f6 after 1 attempt(s): Failed to write to server: 6924a6cc6c904e24bb17911b10b321df (127.14.61.1:39327): Failed to write to server: 6924a6cc6c904e24bb17911b10b321df (127.14.61.1:39327): transaction ID 10 not found, current highest txn ID: 0
W20250111 09:03:03.492630 23255 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40504: Illegal state: Failed to write to server: 6924a6cc6c904e24bb17911b10b321df (127.14.61.1:39327): transaction ID 1 not open: COMMITTED
I20250111 09:03:03.493634 14580 tablet_server.cc:178] TabletServer@127.14.61.1:0 shutting down...
I20250111 09:03:03.496457 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250111 09:03:03.496690 14580 tablet_replica.cc:331] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df: stopping tablet replica
I20250111 09:03:03.496804 14580 raft_consensus.cc:2238] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:03.496893 14580 raft_consensus.cc:2267] T 3b3b19dae82440a2ab50e34eb5a7e9fa P 6924a6cc6c904e24bb17911b10b321df [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:03.497363 14580 tablet_replica.cc:331] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df: stopping tablet replica
I20250111 09:03:03.497447 14580 raft_consensus.cc:2238] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:03.497534 14580 raft_consensus.cc:2267] T bbb8dc5b33d44453be186453374370f6 P 6924a6cc6c904e24bb17911b10b321df [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:03.497789 14580 tablet_replica.cc:331] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df: stopping tablet replica
I20250111 09:03:03.497862 14580 raft_consensus.cc:2238] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:03.497944 14580 raft_consensus.cc:2267] T 6226b69870b945678845fc9ea1f9c188 P 6924a6cc6c904e24bb17911b10b321df [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:03.510465 14580 tablet_server.cc:195] TabletServer@127.14.61.1:0 shutdown complete.
I20250111 09:03:03.512621 14580 master.cc:537] Master@127.14.61.62:42337 shutting down...
I20250111 09:03:03.515625 14580 raft_consensus.cc:2238] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:03.515722 14580 raft_consensus.cc:2267] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:03.515780 14580 tablet_replica.cc:331] T 00000000000000000000000000000000 P a08a45538bd341cbb4842219b288b64e: stopping tablet replica
I20250111 09:03:03.527803 14580 master.cc:559] Master@127.14.61.62:42337 shutdown complete.
[ OK ] TxnOpDispatcherITest.ErrorInParticipantRegistration (1188 ms)
[ RUN ] TxnOpDispatcherITest.ErrorInProcessingWriteOp
I20250111 09:03:03.532326 14580 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.61.62:32915
I20250111 09:03:03.532584 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:03.534761 23265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:03.535367 23264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:03.535948 23267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:03.536566 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:03.536672 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:03.536703 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:03.536722 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586183536721 us; error 0 us; skew 500 ppm
I20250111 09:03:03.537452 14580 webserver.cc:458] Webserver started at http://127.14.61.62:37111/ using document root <none> and password file <none>
I20250111 09:03:03.537577 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:03.537617 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:03.537681 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:03.537974 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.ErrorInProcessingWriteOp.1736586070622223-14580-0/minicluster-data/master-0-root/instance:
uuid: "7013f35ba1534f4980e8133ace8a184f"
format_stamp: "Formatted at 2025-01-11 09:03:03 on dist-test-slave-nvbt"
I20250111 09:03:03.539129 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:03.539866 23272 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:03.540061 14580 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250111 09:03:03.540122 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.ErrorInProcessingWriteOp.1736586070622223-14580-0/minicluster-data/master-0-root
uuid: "7013f35ba1534f4980e8133ace8a184f"
format_stamp: "Formatted at 2025-01-11 09:03:03 on dist-test-slave-nvbt"
I20250111 09:03:03.540180 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.ErrorInProcessingWriteOp.1736586070622223-14580-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.ErrorInProcessingWriteOp.1736586070622223-14580-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.ErrorInProcessingWriteOp.1736586070622223-14580-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:03.556864 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:03.557119 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:03.560799 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:32915
I20250111 09:03:03.561630 23334 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:32915 every 8 connection(s)
I20250111 09:03:03.561947 23335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:03.563436 23335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f: Bootstrap starting.
I20250111 09:03:03.564065 23335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:03.564793 23335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f: No bootstrap required, opened a new log
I20250111 09:03:03.565129 23335 raft_consensus.cc:357] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7013f35ba1534f4980e8133ace8a184f" member_type: VOTER }
I20250111 09:03:03.565202 23335 raft_consensus.cc:383] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:03.565224 23335 raft_consensus.cc:738] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7013f35ba1534f4980e8133ace8a184f, State: Initialized, Role: FOLLOWER
I20250111 09:03:03.565351 23335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7013f35ba1534f4980e8133ace8a184f" member_type: VOTER }
I20250111 09:03:03.565410 23335 raft_consensus.cc:397] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:03.565445 23335 raft_consensus.cc:491] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:03.565495 23335 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:03.566084 23335 raft_consensus.cc:513] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7013f35ba1534f4980e8133ace8a184f" member_type: VOTER }
I20250111 09:03:03.566182 23335 leader_election.cc:304] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7013f35ba1534f4980e8133ace8a184f; no voters:
I20250111 09:03:03.566352 23335 leader_election.cc:290] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:03.566473 23338 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:03.566644 23338 raft_consensus.cc:695] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f [term 1 LEADER]: Becoming Leader. State: Replica: 7013f35ba1534f4980e8133ace8a184f, State: Running, Role: LEADER
I20250111 09:03:03.566663 23335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:03:03.566769 23338 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7013f35ba1534f4980e8133ace8a184f" member_type: VOTER }
I20250111 09:03:03.567158 23339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7013f35ba1534f4980e8133ace8a184f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7013f35ba1534f4980e8133ace8a184f" member_type: VOTER } }
I20250111 09:03:03.567210 23340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7013f35ba1534f4980e8133ace8a184f. Latest consensus state: current_term: 1 leader_uuid: "7013f35ba1534f4980e8133ace8a184f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7013f35ba1534f4980e8133ace8a184f" member_type: VOTER } }
I20250111 09:03:03.567252 23339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:03.567265 23340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:03.567833 23346 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:03:03.568593 23346 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:03:03.569044 14580 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250111 09:03:03.570467 23346 catalog_manager.cc:1348] Generated new cluster ID: 33d26b239dfd4edc88a213c4fe504b4e
I20250111 09:03:03.570520 23346 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:03:03.577540 23346 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:03:03.577981 23346 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:03:03.589848 23346 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f: Generated new TSK 0
I20250111 09:03:03.589969 23346 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:03:03.600524 23289 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49098:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:03.601112 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:03.603292 23363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:03.604012 23365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:03.604947 23362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:03.605882 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:03.606034 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:03.606076 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:03.606097 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586183606097 us; error 0 us; skew 500 ppm
I20250111 09:03:03.607035 14580 webserver.cc:458] Webserver started at http://127.14.61.1:36877/ using document root <none> and password file <none>
I20250111 09:03:03.607172 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:03.607229 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:03.607300 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:03.607638 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.ErrorInProcessingWriteOp.1736586070622223-14580-0/minicluster-data/ts-0-root/instance:
uuid: "1e3545c880334feba152b3d3b0cc8a34"
format_stamp: "Formatted at 2025-01-11 09:03:03 on dist-test-slave-nvbt"
I20250111 09:03:03.608868 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250111 09:03:03.609683 23370 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:03.609866 14580 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250111 09:03:03.609928 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.ErrorInProcessingWriteOp.1736586070622223-14580-0/minicluster-data/ts-0-root
uuid: "1e3545c880334feba152b3d3b0cc8a34"
format_stamp: "Formatted at 2025-01-11 09:03:03 on dist-test-slave-nvbt"
I20250111 09:03:03.609984 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.ErrorInProcessingWriteOp.1736586070622223-14580-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.ErrorInProcessingWriteOp.1736586070622223-14580-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.ErrorInProcessingWriteOp.1736586070622223-14580-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:03.614833 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:03.615113 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:03.615761 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:03:03.615820 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:03.615859 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:03:03.615888 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:03.620607 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:37293
I20250111 09:03:03.620666 23440 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:37293 every 8 connection(s)
I20250111 09:03:03.621459 23441 heartbeater.cc:346] Connected to a master server at 127.14.61.62:32915
I20250111 09:03:03.621536 23441 heartbeater.cc:463] Registering TS with master...
I20250111 09:03:03.621692 23441 heartbeater.cc:510] Master 127.14.61.62:32915 requested a full tablet report, sending...
I20250111 09:03:03.622208 23289 ts_manager.cc:194] Registered new tserver with Master: 1e3545c880334feba152b3d3b0cc8a34 (127.14.61.1:37293)
I20250111 09:03:03.623041 23289 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49106
I20250111 09:03:03.623205 14580 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002255952s
I20250111 09:03:03.628891 23289 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49130:
name: "txn_op_dispatcher_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
I20250111 09:03:03.635671 23399 tablet_service.cc:1467] Processing CreateTablet for tablet 1509cec2061241ef9d2e1f4297319c91 (DEFAULT_TABLE table=txn_op_dispatcher_test [id=5347a0cda9b448488a11f7474b4ccf58]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:03.635671 23400 tablet_service.cc:1467] Processing CreateTablet for tablet b1a4e0fdff2f4304a08503d0f30cd0af (DEFAULT_TABLE table=txn_op_dispatcher_test [id=5347a0cda9b448488a11f7474b4ccf58]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:03.635984 23400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1a4e0fdff2f4304a08503d0f30cd0af. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:03.636143 23399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1509cec2061241ef9d2e1f4297319c91. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:03.638283 23451 tablet_bootstrap.cc:492] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34: Bootstrap starting.
I20250111 09:03:03.639150 23451 tablet_bootstrap.cc:654] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:03.640079 23451 tablet_bootstrap.cc:492] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34: No bootstrap required, opened a new log
I20250111 09:03:03.640166 23451 ts_tablet_manager.cc:1397] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:03.640530 23451 raft_consensus.cc:357] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3545c880334feba152b3d3b0cc8a34" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37293 } }
I20250111 09:03:03.640609 23451 raft_consensus.cc:383] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:03.640647 23451 raft_consensus.cc:738] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e3545c880334feba152b3d3b0cc8a34, State: Initialized, Role: FOLLOWER
I20250111 09:03:03.640767 23451 consensus_queue.cc:260] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3545c880334feba152b3d3b0cc8a34" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37293 } }
I20250111 09:03:03.640830 23451 raft_consensus.cc:397] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:03.640865 23451 raft_consensus.cc:491] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:03.640914 23451 raft_consensus.cc:3054] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:03.641607 23451 raft_consensus.cc:513] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3545c880334feba152b3d3b0cc8a34" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37293 } }
I20250111 09:03:03.641716 23451 leader_election.cc:304] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e3545c880334feba152b3d3b0cc8a34; no voters:
I20250111 09:03:03.641889 23451 leader_election.cc:290] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:03.641950 23453 raft_consensus.cc:2798] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:03.642120 23453 raft_consensus.cc:695] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34 [term 1 LEADER]: Becoming Leader. State: Replica: 1e3545c880334feba152b3d3b0cc8a34, State: Running, Role: LEADER
I20250111 09:03:03.642218 23451 ts_tablet_manager.cc:1428] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250111 09:03:03.642273 23453 consensus_queue.cc:237] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3545c880334feba152b3d3b0cc8a34" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37293 } }
I20250111 09:03:03.642330 23441 heartbeater.cc:502] Master 127.14.61.62:32915 was elected leader, sending a full tablet report...
I20250111 09:03:03.642407 23451 tablet_bootstrap.cc:492] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34: Bootstrap starting.
I20250111 09:03:03.643416 23451 tablet_bootstrap.cc:654] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:03.643416 23289 catalog_manager.cc:5526] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e3545c880334feba152b3d3b0cc8a34 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "1e3545c880334feba152b3d3b0cc8a34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3545c880334feba152b3d3b0cc8a34" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37293 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:03.644374 23451 tablet_bootstrap.cc:492] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34: No bootstrap required, opened a new log
I20250111 09:03:03.644457 23451 ts_tablet_manager.cc:1397] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250111 09:03:03.644922 23451 raft_consensus.cc:357] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3545c880334feba152b3d3b0cc8a34" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37293 } }
I20250111 09:03:03.645036 23451 raft_consensus.cc:383] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:03.645094 23451 raft_consensus.cc:738] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e3545c880334feba152b3d3b0cc8a34, State: Initialized, Role: FOLLOWER
I20250111 09:03:03.645277 23451 consensus_queue.cc:260] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3545c880334feba152b3d3b0cc8a34" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37293 } }
I20250111 09:03:03.645362 23451 raft_consensus.cc:397] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:03.645404 23451 raft_consensus.cc:491] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:03.645452 23451 raft_consensus.cc:3054] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:03.646217 23451 raft_consensus.cc:513] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3545c880334feba152b3d3b0cc8a34" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37293 } }
I20250111 09:03:03.646345 23451 leader_election.cc:304] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e3545c880334feba152b3d3b0cc8a34; no voters:
I20250111 09:03:03.646423 23451 leader_election.cc:290] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:03.646502 23454 raft_consensus.cc:2798] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:03.646585 23454 raft_consensus.cc:695] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34 [term 1 LEADER]: Becoming Leader. State: Replica: 1e3545c880334feba152b3d3b0cc8a34, State: Running, Role: LEADER
I20250111 09:03:03.646647 23451 ts_tablet_manager.cc:1428] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250111 09:03:03.646703 23454 consensus_queue.cc:237] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3545c880334feba152b3d3b0cc8a34" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37293 } }
I20250111 09:03:03.647735 23288 catalog_manager.cc:5526] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e3545c880334feba152b3d3b0cc8a34 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "1e3545c880334feba152b3d3b0cc8a34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3545c880334feba152b3d3b0cc8a34" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37293 } health_report { overall_health: HEALTHY } } }
W20250111 09:03:03.651449 23446 txn_manager_proxy_rpc.cc:150] re-attempting BeginTransaction request to one of TxnManagers
I20250111 09:03:04.608378 23288 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49138:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:04.612905 23399 tablet_service.cc:1467] Processing CreateTablet for tablet 038efaf3816d485baba5b94705c51f6d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1edcff1c29b64d6ca25bd294286ef3c3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:03:04.613198 23399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 038efaf3816d485baba5b94705c51f6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:04.614929 23469 tablet_bootstrap.cc:492] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34: Bootstrap starting.
I20250111 09:03:04.615723 23469 tablet_bootstrap.cc:654] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:04.616586 23469 tablet_bootstrap.cc:492] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34: No bootstrap required, opened a new log
I20250111 09:03:04.616652 23469 ts_tablet_manager.cc:1397] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:04.616961 23469 raft_consensus.cc:357] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3545c880334feba152b3d3b0cc8a34" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37293 } }
I20250111 09:03:04.617074 23469 raft_consensus.cc:383] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:04.617120 23469 raft_consensus.cc:738] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e3545c880334feba152b3d3b0cc8a34, State: Initialized, Role: FOLLOWER
I20250111 09:03:04.617234 23469 consensus_queue.cc:260] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3545c880334feba152b3d3b0cc8a34" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37293 } }
I20250111 09:03:04.617300 23469 raft_consensus.cc:397] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:04.617339 23469 raft_consensus.cc:491] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:04.617383 23469 raft_consensus.cc:3054] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:04.617990 23469 raft_consensus.cc:513] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3545c880334feba152b3d3b0cc8a34" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37293 } }
I20250111 09:03:04.618104 23469 leader_election.cc:304] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e3545c880334feba152b3d3b0cc8a34; no voters:
I20250111 09:03:04.618274 23469 leader_election.cc:290] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:04.618352 23471 raft_consensus.cc:2798] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:04.618496 23471 raft_consensus.cc:695] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34 [term 1 LEADER]: Becoming Leader. State: Replica: 1e3545c880334feba152b3d3b0cc8a34, State: Running, Role: LEADER
I20250111 09:03:04.618583 23469 ts_tablet_manager.cc:1428] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:04.618642 23471 consensus_queue.cc:237] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3545c880334feba152b3d3b0cc8a34" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37293 } }
I20250111 09:03:04.619011 23473 tablet_replica.cc:440] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e3545c880334feba152b3d3b0cc8a34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3545c880334feba152b3d3b0cc8a34" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37293 } } }
I20250111 09:03:04.619019 23472 tablet_replica.cc:440] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34: TxnStatusTablet state changed. Reason: New leader 1e3545c880334feba152b3d3b0cc8a34. Latest consensus state: current_term: 1 leader_uuid: "1e3545c880334feba152b3d3b0cc8a34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3545c880334feba152b3d3b0cc8a34" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37293 } } }
I20250111 09:03:04.619190 23473 tablet_replica.cc:443] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:04.619289 23472 tablet_replica.cc:443] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:04.619597 23475 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:03:04.619674 23475 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:03:04.619805 23288 catalog_manager.cc:5526] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e3545c880334feba152b3d3b0cc8a34 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "1e3545c880334feba152b3d3b0cc8a34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3545c880334feba152b3d3b0cc8a34" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37293 } health_report { overall_health: HEALTHY } } }
W20250111 09:03:04.686250 23476 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45194: Not implemented: transactions may only insert
W20250111 09:03:04.686607 23446 batcher.cc:441] Not implemented: Failed to write batch of 1 ops to tablet b1a4e0fdff2f4304a08503d0f30cd0af after 1 attempt(s): Failed to write to server: 1e3545c880334feba152b3d3b0cc8a34 (127.14.61.1:37293): transactions may only insert
I20250111 09:03:04.719069 14580 tablet_server.cc:178] TabletServer@127.14.61.1:0 shutting down...
I20250111 09:03:04.723088 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250111 09:03:04.723397 14580 tablet_replica.cc:331] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34: stopping tablet replica
I20250111 09:03:04.723527 14580 raft_consensus.cc:2238] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:04.723630 14580 raft_consensus.cc:2267] T 038efaf3816d485baba5b94705c51f6d P 1e3545c880334feba152b3d3b0cc8a34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:04.724162 14580 tablet_replica.cc:331] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34: stopping tablet replica
I20250111 09:03:04.724251 14580 raft_consensus.cc:2238] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:04.724337 14580 raft_consensus.cc:2267] T 1509cec2061241ef9d2e1f4297319c91 P 1e3545c880334feba152b3d3b0cc8a34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:04.724778 14580 tablet_replica.cc:331] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34: stopping tablet replica
I20250111 09:03:04.724856 14580 raft_consensus.cc:2238] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:04.724941 14580 raft_consensus.cc:2267] T b1a4e0fdff2f4304a08503d0f30cd0af P 1e3545c880334feba152b3d3b0cc8a34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:04.737936 14580 tablet_server.cc:195] TabletServer@127.14.61.1:0 shutdown complete.
I20250111 09:03:04.740289 14580 master.cc:537] Master@127.14.61.62:32915 shutting down...
I20250111 09:03:04.745172 14580 raft_consensus.cc:2238] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:04.745290 14580 raft_consensus.cc:2267] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:04.745352 14580 tablet_replica.cc:331] T 00000000000000000000000000000000 P 7013f35ba1534f4980e8133ace8a184f: stopping tablet replica
I20250111 09:03:04.747370 14580 master.cc:559] Master@127.14.61.62:32915 shutdown complete.
[ OK ] TxnOpDispatcherITest.ErrorInProcessingWriteOp (1219 ms)
[ RUN ] TxnOpDispatcherITest.NoPendingWriteOps
I20250111 09:03:04.752096 14580 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.61.62:38963
I20250111 09:03:04.752421 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:04.755417 23494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:04.756081 23492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:04.756767 14580 server_base.cc:1034] running on GCE node
W20250111 09:03:04.756801 23491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:04.756959 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:04.757015 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:04.757035 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586184757034 us; error 0 us; skew 500 ppm
I20250111 09:03:04.757894 14580 webserver.cc:458] Webserver started at http://127.14.61.62:35831/ using document root <none> and password file <none>
I20250111 09:03:04.758021 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:04.758065 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:04.758124 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:04.758476 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.NoPendingWriteOps.1736586070622223-14580-0/minicluster-data/master-0-root/instance:
uuid: "05c55f978de14a6e8ca64db95a3c09b1"
format_stamp: "Formatted at 2025-01-11 09:03:04 on dist-test-slave-nvbt"
I20250111 09:03:04.760023 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250111 09:03:04.760949 23499 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:04.761269 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:04.761353 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.NoPendingWriteOps.1736586070622223-14580-0/minicluster-data/master-0-root
uuid: "05c55f978de14a6e8ca64db95a3c09b1"
format_stamp: "Formatted at 2025-01-11 09:03:04 on dist-test-slave-nvbt"
I20250111 09:03:04.761418 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.NoPendingWriteOps.1736586070622223-14580-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.NoPendingWriteOps.1736586070622223-14580-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.NoPendingWriteOps.1736586070622223-14580-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:04.768209 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:04.768507 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:04.773452 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:38963
I20250111 09:03:04.774282 23561 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:38963 every 8 connection(s)
I20250111 09:03:04.774616 23562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:04.776470 23562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1: Bootstrap starting.
I20250111 09:03:04.777253 23562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:04.778075 23562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1: No bootstrap required, opened a new log
I20250111 09:03:04.778424 23562 raft_consensus.cc:357] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05c55f978de14a6e8ca64db95a3c09b1" member_type: VOTER }
I20250111 09:03:04.778502 23562 raft_consensus.cc:383] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:04.778537 23562 raft_consensus.cc:738] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05c55f978de14a6e8ca64db95a3c09b1, State: Initialized, Role: FOLLOWER
I20250111 09:03:04.778638 23562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05c55f978de14a6e8ca64db95a3c09b1" member_type: VOTER }
I20250111 09:03:04.778702 23562 raft_consensus.cc:397] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:04.778735 23562 raft_consensus.cc:491] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:04.778776 23562 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:04.779556 23562 raft_consensus.cc:513] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05c55f978de14a6e8ca64db95a3c09b1" member_type: VOTER }
I20250111 09:03:04.779678 23562 leader_election.cc:304] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05c55f978de14a6e8ca64db95a3c09b1; no voters:
I20250111 09:03:04.779850 23562 leader_election.cc:290] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:04.779924 23565 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:04.780097 23565 raft_consensus.cc:695] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1 [term 1 LEADER]: Becoming Leader. State: Replica: 05c55f978de14a6e8ca64db95a3c09b1, State: Running, Role: LEADER
I20250111 09:03:04.780205 23565 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05c55f978de14a6e8ca64db95a3c09b1" member_type: VOTER }
I20250111 09:03:04.780242 23562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1 [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:03:04.780539 23565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05c55f978de14a6e8ca64db95a3c09b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05c55f978de14a6e8ca64db95a3c09b1" member_type: VOTER } }
I20250111 09:03:04.780678 23565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1 [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:04.780872 23565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05c55f978de14a6e8ca64db95a3c09b1. Latest consensus state: current_term: 1 leader_uuid: "05c55f978de14a6e8ca64db95a3c09b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05c55f978de14a6e8ca64db95a3c09b1" member_type: VOTER } }
I20250111 09:03:04.780987 23565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1 [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:04.781111 23571 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:03:04.781844 23571 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:03:04.781991 14580 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250111 09:03:04.786527 23571 catalog_manager.cc:1348] Generated new cluster ID: 4d3a588e85cc42eda6bc016d46885bcd
I20250111 09:03:04.786569 23571 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:03:04.802978 23571 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:03:04.803812 23571 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:03:04.810195 23571 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1: Generated new TSK 0
I20250111 09:03:04.810328 23571 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:03:04.813156 23516 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33520:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:04.814390 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:04.816663 23590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:04.817340 23592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:04.818143 23589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:04.818987 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:04.819151 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:04.819195 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:04.819240 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586184819240 us; error 0 us; skew 500 ppm
I20250111 09:03:04.820132 14580 webserver.cc:458] Webserver started at http://127.14.61.1:37127/ using document root <none> and password file <none>
I20250111 09:03:04.820267 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:04.820309 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:04.820379 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:04.820668 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.NoPendingWriteOps.1736586070622223-14580-0/minicluster-data/ts-0-root/instance:
uuid: "f967a13dd6cf428cbe3e8dcefbb08302"
format_stamp: "Formatted at 2025-01-11 09:03:04 on dist-test-slave-nvbt"
I20250111 09:03:04.821919 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250111 09:03:04.822786 23597 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:04.823030 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20250111 09:03:04.823087 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.NoPendingWriteOps.1736586070622223-14580-0/minicluster-data/ts-0-root
uuid: "f967a13dd6cf428cbe3e8dcefbb08302"
format_stamp: "Formatted at 2025-01-11 09:03:04 on dist-test-slave-nvbt"
I20250111 09:03:04.823150 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.NoPendingWriteOps.1736586070622223-14580-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.NoPendingWriteOps.1736586070622223-14580-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.NoPendingWriteOps.1736586070622223-14580-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:04.831368 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:04.831645 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:04.832406 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:03:04.832499 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:04.832590 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:03:04.832659 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:04.837509 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:42167
I20250111 09:03:04.837534 23667 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:42167 every 8 connection(s)
I20250111 09:03:04.838467 23668 heartbeater.cc:346] Connected to a master server at 127.14.61.62:38963
I20250111 09:03:04.838544 23668 heartbeater.cc:463] Registering TS with master...
I20250111 09:03:04.838706 23668 heartbeater.cc:510] Master 127.14.61.62:38963 requested a full tablet report, sending...
I20250111 09:03:04.839314 23516 ts_manager.cc:194] Registered new tserver with Master: f967a13dd6cf428cbe3e8dcefbb08302 (127.14.61.1:42167)
I20250111 09:03:04.840052 23516 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44938
I20250111 09:03:04.840245 14580 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002218484s
I20250111 09:03:04.846727 23516 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44960:
name: "txn_op_dispatcher_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
I20250111 09:03:04.854791 23627 tablet_service.cc:1467] Processing CreateTablet for tablet 10de76b2efa740a9ab7b5a610817edc7 (DEFAULT_TABLE table=txn_op_dispatcher_test [id=2bf9a25f95ff4f2db52212aa527421f3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:04.854915 23626 tablet_service.cc:1467] Processing CreateTablet for tablet 89ceff2d6ca340a08f5ed0063c736a96 (DEFAULT_TABLE table=txn_op_dispatcher_test [id=2bf9a25f95ff4f2db52212aa527421f3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:04.855062 23627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10de76b2efa740a9ab7b5a610817edc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:04.855186 23626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89ceff2d6ca340a08f5ed0063c736a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:04.857102 23678 tablet_bootstrap.cc:492] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302: Bootstrap starting.
I20250111 09:03:04.857957 23678 tablet_bootstrap.cc:654] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:04.858875 23678 tablet_bootstrap.cc:492] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302: No bootstrap required, opened a new log
I20250111 09:03:04.858994 23678 ts_tablet_manager.cc:1397] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:04.859472 23678 raft_consensus.cc:357] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42167 } }
I20250111 09:03:04.859571 23678 raft_consensus.cc:383] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:04.859607 23678 raft_consensus.cc:738] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f967a13dd6cf428cbe3e8dcefbb08302, State: Initialized, Role: FOLLOWER
I20250111 09:03:04.859727 23678 consensus_queue.cc:260] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42167 } }
I20250111 09:03:04.859820 23678 raft_consensus.cc:397] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:04.859872 23678 raft_consensus.cc:491] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:04.859951 23678 raft_consensus.cc:3054] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:04.860630 23678 raft_consensus.cc:513] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42167 } }
I20250111 09:03:04.860747 23678 leader_election.cc:304] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f967a13dd6cf428cbe3e8dcefbb08302; no voters:
I20250111 09:03:04.860908 23678 leader_election.cc:290] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:04.861022 23680 raft_consensus.cc:2798] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:04.861179 23680 raft_consensus.cc:695] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302 [term 1 LEADER]: Becoming Leader. State: Replica: f967a13dd6cf428cbe3e8dcefbb08302, State: Running, Role: LEADER
I20250111 09:03:04.861255 23678 ts_tablet_manager.cc:1428] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:04.861482 23678 tablet_bootstrap.cc:492] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302: Bootstrap starting.
I20250111 09:03:04.861429 23680 consensus_queue.cc:237] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42167 } }
I20250111 09:03:04.862310 23678 tablet_bootstrap.cc:654] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:04.861279 23668 heartbeater.cc:502] Master 127.14.61.62:38963 was elected leader, sending a full tablet report...
I20250111 09:03:04.863379 23678 tablet_bootstrap.cc:492] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302: No bootstrap required, opened a new log
I20250111 09:03:04.863482 23678 ts_tablet_manager.cc:1397] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20250111 09:03:04.863906 23678 raft_consensus.cc:357] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42167 } }
I20250111 09:03:04.863986 23678 raft_consensus.cc:383] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:04.864028 23678 raft_consensus.cc:738] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f967a13dd6cf428cbe3e8dcefbb08302, State: Initialized, Role: FOLLOWER
I20250111 09:03:04.864121 23678 consensus_queue.cc:260] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42167 } }
I20250111 09:03:04.864269 23678 raft_consensus.cc:397] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:04.864320 23678 raft_consensus.cc:491] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:04.864372 23678 raft_consensus.cc:3054] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:04.864934 23678 raft_consensus.cc:513] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42167 } }
I20250111 09:03:04.865082 23678 leader_election.cc:304] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f967a13dd6cf428cbe3e8dcefbb08302; no voters:
I20250111 09:03:04.865245 23678 leader_election.cc:290] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:04.865288 23680 raft_consensus.cc:2798] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:04.865375 23680 raft_consensus.cc:695] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302 [term 1 LEADER]: Becoming Leader. State: Replica: f967a13dd6cf428cbe3e8dcefbb08302, State: Running, Role: LEADER
I20250111 09:03:04.865546 23678 ts_tablet_manager.cc:1428] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250111 09:03:04.865515 23680 consensus_queue.cc:237] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42167 } }
I20250111 09:03:04.864115 23515 catalog_manager.cc:5526] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302 reported cstate change: term changed from 0 to 1, leader changed from <none> to f967a13dd6cf428cbe3e8dcefbb08302 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42167 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:04.868577 23515 catalog_manager.cc:5526] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302 reported cstate change: term changed from 0 to 1, leader changed from <none> to f967a13dd6cf428cbe3e8dcefbb08302 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42167 } health_report { overall_health: HEALTHY } } }
W20250111 09:03:04.876065 23672 txn_manager_proxy_rpc.cc:150] re-attempting BeginTransaction request to one of TxnManagers
I20250111 09:03:05.820235 23515 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44970:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:05.824685 23626 tablet_service.cc:1467] Processing CreateTablet for tablet fb73700a46324e649b5a1f1043dcdf5b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=04529661c8704a5584f46fd4c51379e5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:03:05.824919 23626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb73700a46324e649b5a1f1043dcdf5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:05.826772 23696 tablet_bootstrap.cc:492] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302: Bootstrap starting.
I20250111 09:03:05.827561 23696 tablet_bootstrap.cc:654] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:05.828408 23696 tablet_bootstrap.cc:492] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302: No bootstrap required, opened a new log
I20250111 09:03:05.828483 23696 ts_tablet_manager.cc:1397] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:05.828791 23696 raft_consensus.cc:357] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42167 } }
I20250111 09:03:05.828866 23696 raft_consensus.cc:383] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:05.828897 23696 raft_consensus.cc:738] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f967a13dd6cf428cbe3e8dcefbb08302, State: Initialized, Role: FOLLOWER
I20250111 09:03:05.829031 23696 consensus_queue.cc:260] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42167 } }
I20250111 09:03:05.829107 23696 raft_consensus.cc:397] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:05.829149 23696 raft_consensus.cc:491] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:05.829195 23696 raft_consensus.cc:3054] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:05.829795 23696 raft_consensus.cc:513] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42167 } }
I20250111 09:03:05.829907 23696 leader_election.cc:304] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f967a13dd6cf428cbe3e8dcefbb08302; no voters:
I20250111 09:03:05.830075 23696 leader_election.cc:290] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:05.830133 23698 raft_consensus.cc:2798] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:05.830276 23698 raft_consensus.cc:695] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302 [term 1 LEADER]: Becoming Leader. State: Replica: f967a13dd6cf428cbe3e8dcefbb08302, State: Running, Role: LEADER
I20250111 09:03:05.830386 23696 ts_tablet_manager.cc:1428] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:05.830439 23698 consensus_queue.cc:237] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42167 } }
I20250111 09:03:05.830773 23699 tablet_replica.cc:440] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42167 } } }
I20250111 09:03:05.830902 23699 tablet_replica.cc:443] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:05.830852 23700 tablet_replica.cc:440] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302: TxnStatusTablet state changed. Reason: New leader f967a13dd6cf428cbe3e8dcefbb08302. Latest consensus state: current_term: 1 leader_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42167 } } }
I20250111 09:03:05.831058 23700 tablet_replica.cc:443] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:05.831382 23702 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:03:05.831467 23702 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:03:05.831743 23515 catalog_manager.cc:5526] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302 reported cstate change: term changed from 0 to 1, leader changed from <none> to f967a13dd6cf428cbe3e8dcefbb08302 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a13dd6cf428cbe3e8dcefbb08302" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 42167 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:06.489571 14580 tablet_server.cc:178] TabletServer@127.14.61.1:0 shutting down...
I20250111 09:03:06.492825 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250111 09:03:06.493078 14580 tablet_replica.cc:331] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302: stopping tablet replica
I20250111 09:03:06.493204 14580 raft_consensus.cc:2238] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:06.493310 14580 raft_consensus.cc:2267] T fb73700a46324e649b5a1f1043dcdf5b P f967a13dd6cf428cbe3e8dcefbb08302 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:06.493791 14580 tablet_replica.cc:331] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302: stopping tablet replica
I20250111 09:03:06.493880 14580 raft_consensus.cc:2238] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:06.493963 14580 raft_consensus.cc:2267] T 10de76b2efa740a9ab7b5a610817edc7 P f967a13dd6cf428cbe3e8dcefbb08302 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:06.494306 14580 tablet_replica.cc:331] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302: stopping tablet replica
I20250111 09:03:06.494391 14580 raft_consensus.cc:2238] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:06.494480 14580 raft_consensus.cc:2267] T 89ceff2d6ca340a08f5ed0063c736a96 P f967a13dd6cf428cbe3e8dcefbb08302 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:06.507181 14580 tablet_server.cc:195] TabletServer@127.14.61.1:0 shutdown complete.
I20250111 09:03:06.509312 14580 master.cc:537] Master@127.14.61.62:38963 shutting down...
I20250111 09:03:06.512662 14580 raft_consensus.cc:2238] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:06.512760 14580 raft_consensus.cc:2267] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:06.512804 14580 tablet_replica.cc:331] T 00000000000000000000000000000000 P 05c55f978de14a6e8ca64db95a3c09b1: stopping tablet replica
I20250111 09:03:06.524884 14580 master.cc:559] Master@127.14.61.62:38963 shutdown complete.
[ OK ] TxnOpDispatcherITest.NoPendingWriteOps (1777 ms)
[ RUN ] TxnOpDispatcherITest.NonTransactionalWrites
I20250111 09:03:06.529421 14580 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.61.62:45433
I20250111 09:03:06.529670 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:06.531579 23723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:06.532281 23724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:06.533200 23726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:06.533989 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:06.534101 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:06.534133 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:06.534152 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586186534151 us; error 0 us; skew 500 ppm
I20250111 09:03:06.534870 14580 webserver.cc:458] Webserver started at http://127.14.61.62:35977/ using document root <none> and password file <none>
I20250111 09:03:06.534991 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:06.535035 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:06.535097 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:06.535432 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.NonTransactionalWrites.1736586070622223-14580-0/minicluster-data/master-0-root/instance:
uuid: "96bad9a9589143449890aa9adf9cc137"
format_stamp: "Formatted at 2025-01-11 09:03:06 on dist-test-slave-nvbt"
I20250111 09:03:06.536623 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:06.537436 23731 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:06.537653 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20250111 09:03:06.537714 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.NonTransactionalWrites.1736586070622223-14580-0/minicluster-data/master-0-root
uuid: "96bad9a9589143449890aa9adf9cc137"
format_stamp: "Formatted at 2025-01-11 09:03:06 on dist-test-slave-nvbt"
I20250111 09:03:06.537771 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.NonTransactionalWrites.1736586070622223-14580-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.NonTransactionalWrites.1736586070622223-14580-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.NonTransactionalWrites.1736586070622223-14580-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:06.544517 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:06.544746 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:06.548660 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:45433
I20250111 09:03:06.549872 23793 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:45433 every 8 connection(s)
I20250111 09:03:06.550169 23794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:06.551581 23794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137: Bootstrap starting.
I20250111 09:03:06.552210 23794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:06.552994 23794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137: No bootstrap required, opened a new log
I20250111 09:03:06.553376 23794 raft_consensus.cc:357] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bad9a9589143449890aa9adf9cc137" member_type: VOTER }
I20250111 09:03:06.553453 23794 raft_consensus.cc:383] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:06.553514 23794 raft_consensus.cc:738] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96bad9a9589143449890aa9adf9cc137, State: Initialized, Role: FOLLOWER
I20250111 09:03:06.553673 23794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bad9a9589143449890aa9adf9cc137" member_type: VOTER }
I20250111 09:03:06.553742 23794 raft_consensus.cc:397] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:06.553784 23794 raft_consensus.cc:491] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:06.553844 23794 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:06.554397 23794 raft_consensus.cc:513] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bad9a9589143449890aa9adf9cc137" member_type: VOTER }
I20250111 09:03:06.554526 23794 leader_election.cc:304] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96bad9a9589143449890aa9adf9cc137; no voters:
I20250111 09:03:06.554674 23794 leader_election.cc:290] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:06.554759 23797 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:06.554929 23797 raft_consensus.cc:695] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137 [term 1 LEADER]: Becoming Leader. State: Replica: 96bad9a9589143449890aa9adf9cc137, State: Running, Role: LEADER
I20250111 09:03:06.555023 23794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137 [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:03:06.555054 23797 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bad9a9589143449890aa9adf9cc137" member_type: VOTER }
I20250111 09:03:06.555435 23798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96bad9a9589143449890aa9adf9cc137" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bad9a9589143449890aa9adf9cc137" member_type: VOTER } }
I20250111 09:03:06.555481 23799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96bad9a9589143449890aa9adf9cc137. Latest consensus state: current_term: 1 leader_uuid: "96bad9a9589143449890aa9adf9cc137" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bad9a9589143449890aa9adf9cc137" member_type: VOTER } }
I20250111 09:03:06.555634 23799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137 [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:06.555620 23798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137 [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:06.556134 23805 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:03:06.556669 23805 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:03:06.556814 14580 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250111 09:03:06.558295 23805 catalog_manager.cc:1348] Generated new cluster ID: dcee69bf2bb741849ce9618e636684ba
I20250111 09:03:06.558339 23805 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:03:06.563508 23805 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:03:06.563939 23805 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:03:06.567981 23805 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137: Generated new TSK 0
I20250111 09:03:06.568114 23805 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:03:06.572757 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250111 09:03:06.574678 23748 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42884:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20250111 09:03:06.574786 23821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:06.575562 23824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:06.576172 14580 server_base.cc:1034] running on GCE node
W20250111 09:03:06.576704 23822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:06.576867 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:06.576910 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:06.576929 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586186576929 us; error 0 us; skew 500 ppm
I20250111 09:03:06.577656 14580 webserver.cc:458] Webserver started at http://127.14.61.1:40803/ using document root <none> and password file <none>
I20250111 09:03:06.577775 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:06.577816 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:06.577877 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:06.578138 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.NonTransactionalWrites.1736586070622223-14580-0/minicluster-data/ts-0-root/instance:
uuid: "e30b6764b4b8491182c4641a4f7be173"
format_stamp: "Formatted at 2025-01-11 09:03:06 on dist-test-slave-nvbt"
I20250111 09:03:06.579286 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:06.580003 23829 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:06.580178 14580 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250111 09:03:06.580247 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.NonTransactionalWrites.1736586070622223-14580-0/minicluster-data/ts-0-root
uuid: "e30b6764b4b8491182c4641a4f7be173"
format_stamp: "Formatted at 2025-01-11 09:03:06 on dist-test-slave-nvbt"
I20250111 09:03:06.580300 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.NonTransactionalWrites.1736586070622223-14580-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.NonTransactionalWrites.1736586070622223-14580-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.NonTransactionalWrites.1736586070622223-14580-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:06.605185 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:06.605449 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:06.606055 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:03:06.606146 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:06.606209 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:03:06.606297 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:06.611227 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:35263
I20250111 09:03:06.611255 23899 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:35263 every 8 connection(s)
I20250111 09:03:06.612056 23900 heartbeater.cc:346] Connected to a master server at 127.14.61.62:45433
I20250111 09:03:06.612134 23900 heartbeater.cc:463] Registering TS with master...
I20250111 09:03:06.612358 23900 heartbeater.cc:510] Master 127.14.61.62:45433 requested a full tablet report, sending...
I20250111 09:03:06.612934 23748 ts_manager.cc:194] Registered new tserver with Master: e30b6764b4b8491182c4641a4f7be173 (127.14.61.1:35263)
I20250111 09:03:06.613790 23748 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42896
I20250111 09:03:06.613847 14580 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.0022527s
I20250111 09:03:06.619364 23748 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42914:
name: "txn_op_dispatcher_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
I20250111 09:03:06.625948 23857 tablet_service.cc:1467] Processing CreateTablet for tablet 8325eb0759a341ea9c3976f018c68429 (DEFAULT_TABLE table=txn_op_dispatcher_test [id=80e0c9b1bbd94da5a6a5a09495a138f0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:06.625947 23858 tablet_service.cc:1467] Processing CreateTablet for tablet 033872b99e464fa78fcd9b491432869a (DEFAULT_TABLE table=txn_op_dispatcher_test [id=80e0c9b1bbd94da5a6a5a09495a138f0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:06.626281 23857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8325eb0759a341ea9c3976f018c68429. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:06.626473 23858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 033872b99e464fa78fcd9b491432869a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:06.628013 23910 tablet_bootstrap.cc:492] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173: Bootstrap starting.
I20250111 09:03:06.628720 23910 tablet_bootstrap.cc:654] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:06.629508 23910 tablet_bootstrap.cc:492] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173: No bootstrap required, opened a new log
I20250111 09:03:06.629570 23910 ts_tablet_manager.cc:1397] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:06.629844 23910 raft_consensus.cc:357] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30b6764b4b8491182c4641a4f7be173" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35263 } }
I20250111 09:03:06.629911 23910 raft_consensus.cc:383] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:06.629935 23910 raft_consensus.cc:738] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e30b6764b4b8491182c4641a4f7be173, State: Initialized, Role: FOLLOWER
I20250111 09:03:06.630008 23910 consensus_queue.cc:260] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30b6764b4b8491182c4641a4f7be173" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35263 } }
I20250111 09:03:06.630054 23910 raft_consensus.cc:397] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:06.630077 23910 raft_consensus.cc:491] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:06.630106 23910 raft_consensus.cc:3054] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:06.630638 23910 raft_consensus.cc:513] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30b6764b4b8491182c4641a4f7be173" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35263 } }
I20250111 09:03:06.630736 23910 leader_election.cc:304] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e30b6764b4b8491182c4641a4f7be173; no voters:
I20250111 09:03:06.630874 23910 leader_election.cc:290] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:06.630981 23912 raft_consensus.cc:2798] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:06.631155 23910 ts_tablet_manager.cc:1428] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:06.631208 23900 heartbeater.cc:502] Master 127.14.61.62:45433 was elected leader, sending a full tablet report...
I20250111 09:03:06.631353 23910 tablet_bootstrap.cc:492] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173: Bootstrap starting.
I20250111 09:03:06.631171 23912 raft_consensus.cc:695] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173 [term 1 LEADER]: Becoming Leader. State: Replica: e30b6764b4b8491182c4641a4f7be173, State: Running, Role: LEADER
I20250111 09:03:06.631891 23912 consensus_queue.cc:237] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30b6764b4b8491182c4641a4f7be173" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35263 } }
I20250111 09:03:06.631994 23910 tablet_bootstrap.cc:654] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:06.633173 23910 tablet_bootstrap.cc:492] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173: No bootstrap required, opened a new log
I20250111 09:03:06.633240 23910 ts_tablet_manager.cc:1397] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:06.633298 23748 catalog_manager.cc:5526] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173 reported cstate change: term changed from 0 to 1, leader changed from <none> to e30b6764b4b8491182c4641a4f7be173 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "e30b6764b4b8491182c4641a4f7be173" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30b6764b4b8491182c4641a4f7be173" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35263 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:06.633603 23910 raft_consensus.cc:357] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30b6764b4b8491182c4641a4f7be173" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35263 } }
I20250111 09:03:06.633668 23910 raft_consensus.cc:383] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:06.633687 23910 raft_consensus.cc:738] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e30b6764b4b8491182c4641a4f7be173, State: Initialized, Role: FOLLOWER
I20250111 09:03:06.633781 23910 consensus_queue.cc:260] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30b6764b4b8491182c4641a4f7be173" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35263 } }
I20250111 09:03:06.633834 23910 raft_consensus.cc:397] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:06.633854 23910 raft_consensus.cc:491] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:06.633898 23910 raft_consensus.cc:3054] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:06.634552 23910 raft_consensus.cc:513] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30b6764b4b8491182c4641a4f7be173" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35263 } }
I20250111 09:03:06.634649 23910 leader_election.cc:304] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e30b6764b4b8491182c4641a4f7be173; no voters:
I20250111 09:03:06.634732 23910 leader_election.cc:290] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:06.634769 23912 raft_consensus.cc:2798] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:06.634858 23912 raft_consensus.cc:695] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173 [term 1 LEADER]: Becoming Leader. State: Replica: e30b6764b4b8491182c4641a4f7be173, State: Running, Role: LEADER
I20250111 09:03:06.634930 23910 ts_tablet_manager.cc:1428] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20250111 09:03:06.634987 23912 consensus_queue.cc:237] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30b6764b4b8491182c4641a4f7be173" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35263 } }
I20250111 09:03:06.636034 23748 catalog_manager.cc:5526] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173 reported cstate change: term changed from 0 to 1, leader changed from <none> to e30b6764b4b8491182c4641a4f7be173 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "e30b6764b4b8491182c4641a4f7be173" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30b6764b4b8491182c4641a4f7be173" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35263 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:06.648557 14580 tablet_server.cc:178] TabletServer@127.14.61.1:0 shutting down...
I20250111 09:03:06.651141 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250111 09:03:06.651327 14580 tablet_replica.cc:331] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173: stopping tablet replica
I20250111 09:03:06.651433 14580 raft_consensus.cc:2238] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:06.651535 14580 raft_consensus.cc:2267] T 033872b99e464fa78fcd9b491432869a P e30b6764b4b8491182c4641a4f7be173 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:06.652001 14580 tablet_replica.cc:331] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173: stopping tablet replica
I20250111 09:03:06.652091 14580 raft_consensus.cc:2238] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:06.652191 14580 raft_consensus.cc:2267] T 8325eb0759a341ea9c3976f018c68429 P e30b6764b4b8491182c4641a4f7be173 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:06.664785 14580 tablet_server.cc:195] TabletServer@127.14.61.1:0 shutdown complete.
I20250111 09:03:06.666782 14580 master.cc:537] Master@127.14.61.62:45433 shutting down...
I20250111 09:03:07.576442 14580 raft_consensus.cc:2238] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:07.576671 14580 raft_consensus.cc:2267] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:07.576745 14580 tablet_replica.cc:331] T 00000000000000000000000000000000 P 96bad9a9589143449890aa9adf9cc137: stopping tablet replica
I20250111 09:03:07.589244 14580 master.cc:559] Master@127.14.61.62:45433 shutdown complete.
[ OK ] TxnOpDispatcherITest.NonTransactionalWrites (1065 ms)
[ RUN ] TxnOpDispatcherITest.PreliminaryTasksTimeout
I20250111 09:03:07.594688 14580 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.61.62:39015
I20250111 09:03:07.594972 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:07.597055 23921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:07.597687 23924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:07.598376 23922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:07.599174 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:07.599315 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:07.599351 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:07.599371 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586187599371 us; error 0 us; skew 500 ppm
I20250111 09:03:07.600241 14580 webserver.cc:458] Webserver started at http://127.14.61.62:41221/ using document root <none> and password file <none>
I20250111 09:03:07.600369 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:07.600414 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:07.600492 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:07.600824 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.PreliminaryTasksTimeout.1736586070622223-14580-0/minicluster-data/master-0-root/instance:
uuid: "5f394d77d11e4093a7eca9096fc58085"
format_stamp: "Formatted at 2025-01-11 09:03:07 on dist-test-slave-nvbt"
I20250111 09:03:07.602092 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:07.602955 23929 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:07.603230 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:07.603299 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.PreliminaryTasksTimeout.1736586070622223-14580-0/minicluster-data/master-0-root
uuid: "5f394d77d11e4093a7eca9096fc58085"
format_stamp: "Formatted at 2025-01-11 09:03:07 on dist-test-slave-nvbt"
I20250111 09:03:07.603368 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.PreliminaryTasksTimeout.1736586070622223-14580-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.PreliminaryTasksTimeout.1736586070622223-14580-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.PreliminaryTasksTimeout.1736586070622223-14580-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:07.621160 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:07.621415 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:07.625437 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:39015
I20250111 09:03:07.626389 23991 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:39015 every 8 connection(s)
I20250111 09:03:07.626739 23992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:07.628156 23992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085: Bootstrap starting.
I20250111 09:03:07.628844 23992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:07.629652 23992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085: No bootstrap required, opened a new log
I20250111 09:03:07.629966 23992 raft_consensus.cc:357] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f394d77d11e4093a7eca9096fc58085" member_type: VOTER }
I20250111 09:03:07.630033 23992 raft_consensus.cc:383] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:07.630065 23992 raft_consensus.cc:738] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f394d77d11e4093a7eca9096fc58085, State: Initialized, Role: FOLLOWER
I20250111 09:03:07.630177 23992 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f394d77d11e4093a7eca9096fc58085" member_type: VOTER }
I20250111 09:03:07.630235 23992 raft_consensus.cc:397] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:07.630275 23992 raft_consensus.cc:491] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:07.630319 23992 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:07.630877 23992 raft_consensus.cc:513] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f394d77d11e4093a7eca9096fc58085" member_type: VOTER }
I20250111 09:03:07.630978 23992 leader_election.cc:304] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f394d77d11e4093a7eca9096fc58085; no voters:
I20250111 09:03:07.631127 23992 leader_election.cc:290] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:07.631207 23995 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:07.631384 23995 raft_consensus.cc:695] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085 [term 1 LEADER]: Becoming Leader. State: Replica: 5f394d77d11e4093a7eca9096fc58085, State: Running, Role: LEADER
I20250111 09:03:07.631510 23992 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085 [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:03:07.631538 23995 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f394d77d11e4093a7eca9096fc58085" member_type: VOTER }
I20250111 09:03:07.631882 23996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f394d77d11e4093a7eca9096fc58085" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f394d77d11e4093a7eca9096fc58085" member_type: VOTER } }
I20250111 09:03:07.631964 23996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085 [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:07.631911 23997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f394d77d11e4093a7eca9096fc58085. Latest consensus state: current_term: 1 leader_uuid: "5f394d77d11e4093a7eca9096fc58085" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f394d77d11e4093a7eca9096fc58085" member_type: VOTER } }
I20250111 09:03:07.632036 23997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085 [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:07.632247 23999 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:03:07.633080 23999 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:03:07.633688 14580 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250111 09:03:07.634786 23999 catalog_manager.cc:1348] Generated new cluster ID: 839f7d52cbcd48178e18e61e91e5e073
I20250111 09:03:07.634848 23999 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:03:07.648497 23999 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:03:07.648918 23999 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:03:07.657426 23999 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085: Generated new TSK 0
I20250111 09:03:07.657567 23999 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:03:07.662549 23946 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57810:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:07.665745 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:07.667804 24019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:07.668637 24020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:07.669230 24022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:07.669836 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:07.669974 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:07.670010 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:07.670029 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586187670028 us; error 0 us; skew 500 ppm
I20250111 09:03:07.670706 14580 webserver.cc:458] Webserver started at http://127.14.61.1:37929/ using document root <none> and password file <none>
I20250111 09:03:07.670825 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:07.670866 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:07.670943 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:07.671317 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.PreliminaryTasksTimeout.1736586070622223-14580-0/minicluster-data/ts-0-root/instance:
uuid: "0709b77f06c34e928d864761a36f2367"
format_stamp: "Formatted at 2025-01-11 09:03:07 on dist-test-slave-nvbt"
I20250111 09:03:07.672573 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20250111 09:03:07.673425 24027 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:07.673629 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20250111 09:03:07.673703 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.PreliminaryTasksTimeout.1736586070622223-14580-0/minicluster-data/ts-0-root
uuid: "0709b77f06c34e928d864761a36f2367"
format_stamp: "Formatted at 2025-01-11 09:03:07 on dist-test-slave-nvbt"
I20250111 09:03:07.673774 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.PreliminaryTasksTimeout.1736586070622223-14580-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.PreliminaryTasksTimeout.1736586070622223-14580-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.PreliminaryTasksTimeout.1736586070622223-14580-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:07.693778 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:07.694058 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:07.694738 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:03:07.694783 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:07.694830 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:03:07.694865 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:07.700099 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:37877
I20250111 09:03:07.700107 24097 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:37877 every 8 connection(s)
I20250111 09:03:07.700953 24098 heartbeater.cc:346] Connected to a master server at 127.14.61.62:39015
I20250111 09:03:07.701042 24098 heartbeater.cc:463] Registering TS with master...
I20250111 09:03:07.701205 24098 heartbeater.cc:510] Master 127.14.61.62:39015 requested a full tablet report, sending...
I20250111 09:03:07.701704 23946 ts_manager.cc:194] Registered new tserver with Master: 0709b77f06c34e928d864761a36f2367 (127.14.61.1:37877)
I20250111 09:03:07.702673 23946 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57814
I20250111 09:03:07.702693 14580 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002319324s
I20250111 09:03:07.708830 23945 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57836:
name: "txn_op_dispatcher_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
I20250111 09:03:07.715710 24056 tablet_service.cc:1467] Processing CreateTablet for tablet 7ff598f6fb504fbaa196f685fea5a77f (DEFAULT_TABLE table=txn_op_dispatcher_test [id=0fb0df0ad8bc4f3a9b0456c2bc243928]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:07.715804 24057 tablet_service.cc:1467] Processing CreateTablet for tablet 2d43639475a7400aa2ef043781ceca41 (DEFAULT_TABLE table=txn_op_dispatcher_test [id=0fb0df0ad8bc4f3a9b0456c2bc243928]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:07.715952 24056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ff598f6fb504fbaa196f685fea5a77f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:07.716120 24057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d43639475a7400aa2ef043781ceca41. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:07.717698 24108 tablet_bootstrap.cc:492] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367: Bootstrap starting.
I20250111 09:03:07.718482 24108 tablet_bootstrap.cc:654] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:07.719323 24108 tablet_bootstrap.cc:492] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367: No bootstrap required, opened a new log
I20250111 09:03:07.719403 24108 ts_tablet_manager.cc:1397] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:07.719727 24108 raft_consensus.cc:357] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0709b77f06c34e928d864761a36f2367" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37877 } }
I20250111 09:03:07.719800 24108 raft_consensus.cc:383] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:07.719835 24108 raft_consensus.cc:738] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0709b77f06c34e928d864761a36f2367, State: Initialized, Role: FOLLOWER
I20250111 09:03:07.719961 24108 consensus_queue.cc:260] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0709b77f06c34e928d864761a36f2367" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37877 } }
I20250111 09:03:07.720017 24108 raft_consensus.cc:397] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:07.720049 24108 raft_consensus.cc:491] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:07.720093 24108 raft_consensus.cc:3054] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:07.720676 24108 raft_consensus.cc:513] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0709b77f06c34e928d864761a36f2367" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37877 } }
I20250111 09:03:07.720769 24108 leader_election.cc:304] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0709b77f06c34e928d864761a36f2367; no voters:
I20250111 09:03:07.720924 24108 leader_election.cc:290] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:07.721035 24110 raft_consensus.cc:2798] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:07.721212 24108 ts_tablet_manager.cc:1428] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:07.721252 24098 heartbeater.cc:502] Master 127.14.61.62:39015 was elected leader, sending a full tablet report...
I20250111 09:03:07.721369 24110 raft_consensus.cc:695] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367 [term 1 LEADER]: Becoming Leader. State: Replica: 0709b77f06c34e928d864761a36f2367, State: Running, Role: LEADER
I20250111 09:03:07.721374 24108 tablet_bootstrap.cc:492] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367: Bootstrap starting.
I20250111 09:03:07.721518 24110 consensus_queue.cc:237] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0709b77f06c34e928d864761a36f2367" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37877 } }
I20250111 09:03:07.722163 24108 tablet_bootstrap.cc:654] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:07.722646 23945 catalog_manager.cc:5526] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0709b77f06c34e928d864761a36f2367 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "0709b77f06c34e928d864761a36f2367" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0709b77f06c34e928d864761a36f2367" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37877 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:07.723167 24108 tablet_bootstrap.cc:492] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367: No bootstrap required, opened a new log
I20250111 09:03:07.723251 24108 ts_tablet_manager.cc:1397] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250111 09:03:07.723688 24108 raft_consensus.cc:357] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0709b77f06c34e928d864761a36f2367" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37877 } }
I20250111 09:03:07.723784 24108 raft_consensus.cc:383] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:07.723819 24108 raft_consensus.cc:738] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0709b77f06c34e928d864761a36f2367, State: Initialized, Role: FOLLOWER
I20250111 09:03:07.723917 24108 consensus_queue.cc:260] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0709b77f06c34e928d864761a36f2367" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37877 } }
I20250111 09:03:07.723989 24108 raft_consensus.cc:397] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:07.724009 24108 raft_consensus.cc:491] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:07.724033 24108 raft_consensus.cc:3054] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:07.724726 24108 raft_consensus.cc:513] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0709b77f06c34e928d864761a36f2367" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37877 } }
I20250111 09:03:07.724834 24108 leader_election.cc:304] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0709b77f06c34e928d864761a36f2367; no voters:
I20250111 09:03:07.724911 24108 leader_election.cc:290] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:07.724973 24110 raft_consensus.cc:2798] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:07.725142 24108 ts_tablet_manager.cc:1428] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250111 09:03:07.725800 24110 raft_consensus.cc:695] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367 [term 1 LEADER]: Becoming Leader. State: Replica: 0709b77f06c34e928d864761a36f2367, State: Running, Role: LEADER
I20250111 09:03:07.725944 24110 consensus_queue.cc:237] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0709b77f06c34e928d864761a36f2367" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37877 } }
I20250111 09:03:07.727028 23946 catalog_manager.cc:5526] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0709b77f06c34e928d864761a36f2367 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "0709b77f06c34e928d864761a36f2367" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0709b77f06c34e928d864761a36f2367" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37877 } health_report { overall_health: HEALTHY } } }
W20250111 09:03:07.731416 24102 txn_manager_proxy_rpc.cc:150] re-attempting BeginTransaction request to one of TxnManagers
I20250111 09:03:08.670516 23946 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57846:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:08.674980 24057 tablet_service.cc:1467] Processing CreateTablet for tablet 7e30d9f5c9b044f89ec72d8f21698d6e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b4f1e0683ad04999a39e9a25d0ec3b89]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:03:08.675248 24057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e30d9f5c9b044f89ec72d8f21698d6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:08.677119 24126 tablet_bootstrap.cc:492] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367: Bootstrap starting.
I20250111 09:03:08.677872 24126 tablet_bootstrap.cc:654] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:08.678711 24126 tablet_bootstrap.cc:492] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367: No bootstrap required, opened a new log
I20250111 09:03:08.678773 24126 ts_tablet_manager.cc:1397] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250111 09:03:08.679109 24126 raft_consensus.cc:357] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0709b77f06c34e928d864761a36f2367" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37877 } }
I20250111 09:03:08.679183 24126 raft_consensus.cc:383] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:08.679214 24126 raft_consensus.cc:738] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0709b77f06c34e928d864761a36f2367, State: Initialized, Role: FOLLOWER
I20250111 09:03:08.679324 24126 consensus_queue.cc:260] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0709b77f06c34e928d864761a36f2367" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37877 } }
I20250111 09:03:08.679385 24126 raft_consensus.cc:397] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:08.679417 24126 raft_consensus.cc:491] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:08.679469 24126 raft_consensus.cc:3054] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:08.680091 24126 raft_consensus.cc:513] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0709b77f06c34e928d864761a36f2367" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37877 } }
I20250111 09:03:08.680195 24126 leader_election.cc:304] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0709b77f06c34e928d864761a36f2367; no voters:
I20250111 09:03:08.680380 24126 leader_election.cc:290] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:08.680449 24128 raft_consensus.cc:2798] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:08.680588 24128 raft_consensus.cc:695] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367 [term 1 LEADER]: Becoming Leader. State: Replica: 0709b77f06c34e928d864761a36f2367, State: Running, Role: LEADER
I20250111 09:03:08.680732 24126 ts_tablet_manager.cc:1428] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250111 09:03:08.680699 24128 consensus_queue.cc:237] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0709b77f06c34e928d864761a36f2367" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37877 } }
I20250111 09:03:08.681264 24129 tablet_replica.cc:440] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0709b77f06c34e928d864761a36f2367" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0709b77f06c34e928d864761a36f2367" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37877 } } }
I20250111 09:03:08.681388 24129 tablet_replica.cc:443] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:08.681326 24130 tablet_replica.cc:440] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367: TxnStatusTablet state changed. Reason: New leader 0709b77f06c34e928d864761a36f2367. Latest consensus state: current_term: 1 leader_uuid: "0709b77f06c34e928d864761a36f2367" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0709b77f06c34e928d864761a36f2367" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37877 } } }
I20250111 09:03:08.681640 24130 tablet_replica.cc:443] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:08.681964 23946 catalog_manager.cc:5526] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0709b77f06c34e928d864761a36f2367 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "0709b77f06c34e928d864761a36f2367" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0709b77f06c34e928d864761a36f2367" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37877 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:08.682312 24132 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:03:08.682369 24132 txn_status_manager.cc:930] Loading transaction status metadata into memory...
W20250111 09:03:10.240571 24141 meta_cache.cc:302] tablet 2d43639475a7400aa2ef043781ceca41: replica 0709b77f06c34e928d864761a36f2367 (127.14.61.1:37877) has failed: Timed out: Write RPC to 127.14.61.1:37877 timed out after 0.996s (SENT)
W20250111 09:03:10.240681 24141 batcher.cc:441] Timed out: Failed to write batch of 1 ops to tablet 2d43639475a7400aa2ef043781ceca41 after 1 attempt(s): Failed to write to server: 0709b77f06c34e928d864761a36f2367 (127.14.61.1:37877): Write RPC to 127.14.61.1:37877 timed out after 0.996s (SENT)
W20250111 09:03:11.248637 24155 tablet_replica.cc:1307] Timed out: time out prior registering tablet 2d43639475a7400aa2ef043781ceca41 as participant (txn ID 0): cancelling pending write operations [suppressed 1 similar messages]
W20250111 09:03:11.248699 24156 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:38880: Timed out: time out prior registering tablet 7ff598f6fb504fbaa196f685fea5a77f as participant (txn ID 0)
W20250111 09:03:11.248744 24155 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:38880: Timed out: time out prior registering tablet 2d43639475a7400aa2ef043781ceca41 as participant (txn ID 0)
W20250111 09:03:11.248924 24156 rpcz_store.cc:267] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38880 (ReqId={client: feb2cf770daf468f86eba7169c84f6b4, seq_no=1, attempt_no=0}) took 2002 ms (client timeout 998 ms). Trace:
W20250111 09:03:11.248931 24155 rpcz_store.cc:267] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38880 (ReqId={client: feb2cf770daf468f86eba7169c84f6b4, seq_no=0, attempt_no=0}) took 2002 ms (client timeout 999 ms). Trace:
W20250111 09:03:11.248975 24156 rpcz_store.cc:269] 0111 09:03:09.246795 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:09.247148 (+ 353us) service_pool.cc:225] Handling call
0111 09:03:11.248905 (+2001757us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":74,"threads_started":1,"txn-participant-registration.queue_time_us":182}
W20250111 09:03:11.249034 24155 rpcz_store.cc:269] 0111 09:03:09.246711 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:09.246760 (+ 49us) service_pool.cc:225] Handling call
0111 09:03:11.248921 (+2002161us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":74112,"thread_start_us":86,"threads_started":1,"txn-participant-registration.queue_time_us":155}
W20250111 09:03:11.249202 24156 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:38880: Timed out: time out prior registering tablet 7ff598f6fb504fbaa196f685fea5a77f as participant (txn ID 0)
W20250111 09:03:11.249254 24155 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:38880: Timed out: time out prior registering tablet 2d43639475a7400aa2ef043781ceca41 as participant (txn ID 0)
W20250111 09:03:11.249305 24156 rpcz_store.cc:267] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38880 (ReqId={client: feb2cf770daf468f86eba7169c84f6b4, seq_no=2, attempt_no=0}) took 2002 ms (client timeout 998 ms). Trace:
W20250111 09:03:11.249373 24155 rpcz_store.cc:267] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38880 (ReqId={client: feb2cf770daf468f86eba7169c84f6b4, seq_no=4, attempt_no=0}) took 2002 ms (client timeout 998 ms). Trace:
W20250111 09:03:11.249369 24156 rpcz_store.cc:269] 0111 09:03:09.246912 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:09.247588 (+ 676us) service_pool.cc:225] Handling call
0111 09:03:11.249295 (+2001707us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1805056}
W20250111 09:03:11.249447 24155 rpcz_store.cc:269] 0111 09:03:09.246999 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:09.247076 (+ 77us) service_pool.cc:225] Handling call
0111 09:03:09.248656 (+ 1580us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x562c9fb708a0. stack: 00007f22e25c91af 00007f22e25c97a1 00007f22e1b0d89d 0000562c8c26a85e 0000562c8c26a905 0000562c8c27191a 00007f22e3772b8b 00007f22e376c17a 00007f22e43be366 00007f22e4c3ed15 00007f22e4c41a25 00007f22e3126895 00007f22e3126149 00007f22e3128e5a 00007f22e31277a4 00007f22e31291e3
0111 09:03:11.249365 (+2000709us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2748672}
W20250111 09:03:11.249512 24156 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:38880: Timed out: time out prior registering tablet 7ff598f6fb504fbaa196f685fea5a77f as participant (txn ID 0)
W20250111 09:03:11.249567 24155 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:38880: Timed out: time out prior registering tablet 2d43639475a7400aa2ef043781ceca41 as participant (txn ID 0)
W20250111 09:03:11.249601 24156 rpcz_store.cc:267] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38880 (ReqId={client: feb2cf770daf468f86eba7169c84f6b4, seq_no=3, attempt_no=0}) took 2002 ms (client timeout 998 ms). Trace:
W20250111 09:03:11.249665 24156 rpcz_store.cc:269] 0111 09:03:09.246956 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:09.247806 (+ 850us) service_pool.cc:225] Handling call
0111 09:03:11.249594 (+2001788us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1386368}
W20250111 09:03:11.249697 24155 rpcz_store.cc:267] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38880 (ReqId={client: feb2cf770daf468f86eba7169c84f6b4, seq_no=6, attempt_no=0}) took 2002 ms (client timeout 997 ms). Trace:
W20250111 09:03:11.249737 24155 rpcz_store.cc:269] 0111 09:03:09.247104 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:09.247151 (+ 47us) service_pool.cc:225] Handling call
0111 09:03:09.248669 (+ 1518us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x562c9fb708a0. stack: 00007f22e25c91af 00007f22e25c97a1 00007f22e1b0d89d 0000562c8c26a85e 0000562c8c26a905 0000562c8c27191a 00007f22e3772b8b 00007f22e376c17a 00007f22e43be366 00007f22e4c3ed15 00007f22e4c41a25 00007f22e3126895 00007f22e3126149 00007f22e3128e5a 00007f22e31277a4 00007f22e31291e3
0111 09:03:11.249688 (+2001019us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2536064}
W20250111 09:03:11.249825 24155 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:38880: Timed out: time out prior registering tablet 2d43639475a7400aa2ef043781ceca41 as participant (txn ID 0)
W20250111 09:03:11.249912 24156 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:38880: Timed out: time out prior registering tablet 7ff598f6fb504fbaa196f685fea5a77f as participant (txn ID 0)
W20250111 09:03:11.249915 24155 rpcz_store.cc:267] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38880 (ReqId={client: feb2cf770daf468f86eba7169c84f6b4, seq_no=7, attempt_no=0}) took 2003 ms (client timeout 995 ms). Trace:
W20250111 09:03:11.249986 24155 rpcz_store.cc:269] 0111 09:03:09.246849 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:09.247363 (+ 514us) service_pool.cc:225] Handling call
0111 09:03:09.248654 (+ 1291us) spinlock_profiling.cc:243] Waited 990 us on lock 0x562c9fb708a0. stack: 00007f22e25c91af 00007f22e25c97a1 00007f22e1b0d89d 0000562c8c26a85e 0000562c8c26a905 0000562c8c27191a 00007f22e3772b8b 00007f22e376c17a 00007f22e43be366 00007f22e4c3ed15 00007f22e4c41a25 00007f22e3126895 00007f22e3126149 00007f22e3128e5a 00007f22e31277a4 00007f22e31291e3
0111 09:03:11.249909 (+2001255us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2277248}
W20250111 09:03:11.250031 24156 rpcz_store.cc:267] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38880 (ReqId={client: feb2cf770daf468f86eba7169c84f6b4, seq_no=5, attempt_no=0}) took 2003 ms (client timeout 997 ms). Trace:
W20250111 09:03:11.250072 24156 rpcz_store.cc:269] 0111 09:03:09.247050 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:09.247989 (+ 939us) service_pool.cc:225] Handling call
0111 09:03:11.250024 (+2002035us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1008512}
W20250111 09:03:12.290407 24166 meta_cache.cc:302] tablet 2d43639475a7400aa2ef043781ceca41: replica 0709b77f06c34e928d864761a36f2367 (127.14.61.1:37877) has failed: Timed out: Write RPC to 127.14.61.1:37877 timed out after 0.998s (SENT)
W20250111 09:03:12.290530 24166 batcher.cc:441] Timed out: Failed to write batch of 1 ops to tablet 2d43639475a7400aa2ef043781ceca41 after 1 attempt(s): Failed to write to server: 0709b77f06c34e928d864761a36f2367 (127.14.61.1:37877): Write RPC to 127.14.61.1:37877 timed out after 0.998s (SENT)
W20250111 09:03:13.298483 24156 tablet_replica.cc:1307] Timed out: time out prior submitting BEGIN_TXN for participant 2d43639475a7400aa2ef043781ceca41 (txn ID 1): cancelling pending write operations [suppressed 1 similar messages]
W20250111 09:03:13.298588 24156 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:38902: Timed out: time out prior submitting BEGIN_TXN for participant 2d43639475a7400aa2ef043781ceca41 (txn ID 1)
W20250111 09:03:13.298723 24155 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:38902: Timed out: time out prior submitting BEGIN_TXN for participant 7ff598f6fb504fbaa196f685fea5a77f (txn ID 1)
W20250111 09:03:13.298791 24156 rpcz_store.cc:267] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38902 (ReqId={client: 1f6277bcd4c84e05ab2e5ce3b08541ac, seq_no=1, attempt_no=0}) took 2005 ms (client timeout 998 ms). Trace:
W20250111 09:03:13.298905 24155 rpcz_store.cc:267] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38902 (ReqId={client: 1f6277bcd4c84e05ab2e5ce3b08541ac, seq_no=0, attempt_no=0}) took 2005 ms (client timeout 998 ms). Trace:
W20250111 09:03:13.298863 24156 rpcz_store.cc:269] 0111 09:03:11.293659 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:11.293705 (+ 46us) service_pool.cc:225] Handling call
0111 09:03:13.298778 (+2005073us) inbound_call.cc:173] Queueing success response
Metrics: {"txn-participant-registration.queue_time_us":49}
W20250111 09:03:13.298957 24155 rpcz_store.cc:269] 0111 09:03:11.293571 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:11.293616 (+ 45us) service_pool.cc:225] Handling call
0111 09:03:13.298894 (+2005278us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":6144,"txn-participant-registration.queue_time_us":74}
W20250111 09:03:13.299126 24155 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:38902: Timed out: time out prior submitting BEGIN_TXN for participant 7ff598f6fb504fbaa196f685fea5a77f (txn ID 1)
W20250111 09:03:13.299134 24156 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:38902: Timed out: time out prior submitting BEGIN_TXN for participant 2d43639475a7400aa2ef043781ceca41 (txn ID 1)
W20250111 09:03:13.299273 24155 rpcz_store.cc:267] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38902 (ReqId={client: 1f6277bcd4c84e05ab2e5ce3b08541ac, seq_no=4, attempt_no=0}) took 2005 ms (client timeout 998 ms). Trace:
W20250111 09:03:13.299311 24156 rpcz_store.cc:267] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38902 (ReqId={client: 1f6277bcd4c84e05ab2e5ce3b08541ac, seq_no=2, attempt_no=0}) took 2005 ms (client timeout 998 ms). Trace:
W20250111 09:03:13.299322 24155 rpcz_store.cc:269] 0111 09:03:11.293848 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:11.293877 (+ 29us) service_pool.cc:225] Handling call
0111 09:03:13.299263 (+2005386us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20250111 09:03:13.299374 24156 rpcz_store.cc:269] 0111 09:03:11.293727 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:11.293775 (+ 48us) service_pool.cc:225] Handling call
0111 09:03:13.299300 (+2005525us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":19328}
W20250111 09:03:13.299520 24156 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:38902: Timed out: time out prior submitting BEGIN_TXN for participant 2d43639475a7400aa2ef043781ceca41 (txn ID 1)
W20250111 09:03:13.299520 24155 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:38902: Timed out: time out prior submitting BEGIN_TXN for participant 7ff598f6fb504fbaa196f685fea5a77f (txn ID 1)
W20250111 09:03:13.299656 24156 rpcz_store.cc:267] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38902 (ReqId={client: 1f6277bcd4c84e05ab2e5ce3b08541ac, seq_no=3, attempt_no=0}) took 2005 ms (client timeout 998 ms). Trace:
W20250111 09:03:13.299687 24155 rpcz_store.cc:267] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38902 (ReqId={client: 1f6277bcd4c84e05ab2e5ce3b08541ac, seq_no=6, attempt_no=0}) took 2005 ms (client timeout 997 ms). Trace:
W20250111 09:03:13.299711 24156 rpcz_store.cc:269] 0111 09:03:11.293790 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:11.294081 (+ 291us) service_pool.cc:225] Handling call
0111 09:03:13.299647 (+2005566us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":27520}
W20250111 09:03:13.299746 24155 rpcz_store.cc:269] 0111 09:03:11.293979 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:11.294204 (+ 225us) service_pool.cc:225] Handling call
0111 09:03:13.299675 (+2005471us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20250111 09:03:13.299831 24156 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:38902: Timed out: time out prior submitting BEGIN_TXN for participant 2d43639475a7400aa2ef043781ceca41 (txn ID 1)
W20250111 09:03:13.299840 24155 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:38902: Timed out: time out prior submitting BEGIN_TXN for participant 7ff598f6fb504fbaa196f685fea5a77f (txn ID 1)
W20250111 09:03:13.299952 24156 rpcz_store.cc:267] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38902 (ReqId={client: 1f6277bcd4c84e05ab2e5ce3b08541ac, seq_no=7, attempt_no=0}) took 2005 ms (client timeout 997 ms). Trace:
W20250111 09:03:13.299978 24155 rpcz_store.cc:267] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38902 (ReqId={client: 1f6277bcd4c84e05ab2e5ce3b08541ac, seq_no=5, attempt_no=0}) took 2006 ms (client timeout 997 ms). Trace:
W20250111 09:03:13.300042 24155 rpcz_store.cc:269] 0111 09:03:11.293909 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:11.294211 (+ 302us) service_pool.cc:225] Handling call
0111 09:03:13.299967 (+2005756us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20250111 09:03:13.300025 24156 rpcz_store.cc:269] 0111 09:03:11.294046 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:11.294082 (+ 36us) service_pool.cc:225] Handling call
0111 09:03:13.299944 (+2005862us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":15104}
W20250111 09:03:13.320302 24182 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:38918: Illegal state: Transaction hasn't been successfully started
I20250111 09:03:13.320667 24080 txn_status_manager.cc:244] Participant 7ff598f6fb504fbaa196f685fea5a77f of txn 1 returned error for BEGIN_COMMIT op, aborting: Illegal state: Failed to write to server: 0709b77f06c34e928d864761a36f2367 (127.14.61.1:37877): Transaction hasn't been successfully started
W20250111 09:03:13.320791 24182 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:38918: Illegal state: Transaction hasn't been successfully started
I20250111 09:03:13.321039 24080 txn_status_manager.cc:244] Participant 2d43639475a7400aa2ef043781ceca41 of txn 1 returned error for BEGIN_COMMIT op, aborting: Illegal state: Failed to write to server: 0709b77f06c34e928d864761a36f2367 (127.14.61.1:37877): Transaction hasn't been successfully started
I20250111 09:03:13.321087 24080 txn_status_manager.cc:206] Scheduling write for ABORT_IN_PROGRESS for txn 1
I20250111 09:03:13.328691 14580 tablet_server.cc:178] TabletServer@127.14.61.1:0 shutting down...
I20250111 09:03:13.331739 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250111 09:03:13.331954 14580 tablet_replica.cc:331] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367: stopping tablet replica
I20250111 09:03:13.332058 14580 raft_consensus.cc:2238] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:13.332170 14580 raft_consensus.cc:2267] T 7e30d9f5c9b044f89ec72d8f21698d6e P 0709b77f06c34e928d864761a36f2367 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:13.332633 14580 tablet_replica.cc:331] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367: stopping tablet replica
I20250111 09:03:13.332702 14580 raft_consensus.cc:2238] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:13.332792 14580 raft_consensus.cc:2267] T 7ff598f6fb504fbaa196f685fea5a77f P 0709b77f06c34e928d864761a36f2367 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:13.333272 14580 tablet_replica.cc:331] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367: stopping tablet replica
I20250111 09:03:13.333345 14580 raft_consensus.cc:2238] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:13.333436 14580 raft_consensus.cc:2267] T 2d43639475a7400aa2ef043781ceca41 P 0709b77f06c34e928d864761a36f2367 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:13.346026 14580 tablet_server.cc:195] TabletServer@127.14.61.1:0 shutdown complete.
I20250111 09:03:13.348019 14580 master.cc:537] Master@127.14.61.62:39015 shutting down...
I20250111 09:03:13.351450 14580 raft_consensus.cc:2238] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:13.351559 14580 raft_consensus.cc:2267] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:13.351620 14580 tablet_replica.cc:331] T 00000000000000000000000000000000 P 5f394d77d11e4093a7eca9096fc58085: stopping tablet replica
I20250111 09:03:13.363322 14580 master.cc:559] Master@127.14.61.62:39015 shutdown complete.
[ OK ] TxnOpDispatcherITest.PreliminaryTasksTimeout (5773 ms)
[ RUN ] TxnOpDispatcherITest.DuplicateTxnParticipantRegistration
I20250111 09:03:13.368112 14580 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.61.62:36935
I20250111 09:03:13.368407 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:13.370296 24191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:13.370942 24194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:13.371783 24192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:13.372567 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:13.372704 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:13.372740 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:13.372756 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586193372754 us; error 0 us; skew 500 ppm
I20250111 09:03:13.377882 14580 webserver.cc:458] Webserver started at http://127.14.61.62:33067/ using document root <none> and password file <none>
I20250111 09:03:13.378027 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:13.378074 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:13.378147 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:13.378556 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.DuplicateTxnParticipantRegistration.1736586070622223-14580-0/minicluster-data/master-0-root/instance:
uuid: "155ea70706fa41d8809872614a81aedd"
format_stamp: "Formatted at 2025-01-11 09:03:13 on dist-test-slave-nvbt"
I20250111 09:03:13.379980 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:13.380805 24199 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:13.381037 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:13.381101 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.DuplicateTxnParticipantRegistration.1736586070622223-14580-0/minicluster-data/master-0-root
uuid: "155ea70706fa41d8809872614a81aedd"
format_stamp: "Formatted at 2025-01-11 09:03:13 on dist-test-slave-nvbt"
I20250111 09:03:13.381151 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.DuplicateTxnParticipantRegistration.1736586070622223-14580-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.DuplicateTxnParticipantRegistration.1736586070622223-14580-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.DuplicateTxnParticipantRegistration.1736586070622223-14580-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:13.390313 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:13.390547 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:13.394488 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:36935
I20250111 09:03:13.395326 24261 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:36935 every 8 connection(s)
I20250111 09:03:13.395669 24262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:13.397202 24262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd: Bootstrap starting.
I20250111 09:03:13.397867 24262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:13.398625 24262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd: No bootstrap required, opened a new log
I20250111 09:03:13.398967 24262 raft_consensus.cc:357] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "155ea70706fa41d8809872614a81aedd" member_type: VOTER }
I20250111 09:03:13.399042 24262 raft_consensus.cc:383] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:13.399080 24262 raft_consensus.cc:738] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 155ea70706fa41d8809872614a81aedd, State: Initialized, Role: FOLLOWER
I20250111 09:03:13.399216 24262 consensus_queue.cc:260] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "155ea70706fa41d8809872614a81aedd" member_type: VOTER }
I20250111 09:03:13.399304 24262 raft_consensus.cc:397] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:13.399344 24262 raft_consensus.cc:491] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:13.399396 24262 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:13.400002 24262 raft_consensus.cc:513] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "155ea70706fa41d8809872614a81aedd" member_type: VOTER }
I20250111 09:03:13.400117 24262 leader_election.cc:304] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 155ea70706fa41d8809872614a81aedd; no voters:
I20250111 09:03:13.400280 24262 leader_election.cc:290] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:13.400341 24265 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:13.400496 24265 raft_consensus.cc:695] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd [term 1 LEADER]: Becoming Leader. State: Replica: 155ea70706fa41d8809872614a81aedd, State: Running, Role: LEADER
I20250111 09:03:13.400640 24262 sys_catalog.cc:564] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:03:13.400638 24265 consensus_queue.cc:237] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "155ea70706fa41d8809872614a81aedd" member_type: VOTER }
I20250111 09:03:13.401042 24267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 155ea70706fa41d8809872614a81aedd. Latest consensus state: current_term: 1 leader_uuid: "155ea70706fa41d8809872614a81aedd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "155ea70706fa41d8809872614a81aedd" member_type: VOTER } }
I20250111 09:03:13.401031 24266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "155ea70706fa41d8809872614a81aedd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "155ea70706fa41d8809872614a81aedd" member_type: VOTER } }
I20250111 09:03:13.401125 24267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:13.401173 24266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:13.402349 14580 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
W20250111 09:03:13.402755 24281 catalog_manager.cc:1559] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250111 09:03:13.402819 24281 catalog_manager.cc:874] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250111 09:03:13.403782 24271 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:03:13.404397 24271 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:03:13.406741 24271 catalog_manager.cc:1348] Generated new cluster ID: b6df5c4e67114205bce354cc3b5c8fc9
I20250111 09:03:13.406814 24271 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:03:13.413075 24271 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:03:13.413530 24271 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:03:13.423038 24271 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd: Generated new TSK 0
I20250111 09:03:13.423182 24271 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:03:13.428583 24216 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33256:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:13.434562 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:13.436655 24290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:13.437161 24289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:13.437729 24292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:13.438575 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:13.438683 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:13.438714 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:13.438728 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586193438728 us; error 0 us; skew 500 ppm
I20250111 09:03:13.439432 14580 webserver.cc:458] Webserver started at http://127.14.61.1:39891/ using document root <none> and password file <none>
I20250111 09:03:13.439554 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:13.439596 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:13.439659 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:13.440048 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.DuplicateTxnParticipantRegistration.1736586070622223-14580-0/minicluster-data/ts-0-root/instance:
uuid: "f40914ef612942d68eb28cbbfa6dac68"
format_stamp: "Formatted at 2025-01-11 09:03:13 on dist-test-slave-nvbt"
I20250111 09:03:13.441211 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250111 09:03:13.441910 24297 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:13.442097 14580 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250111 09:03:13.442162 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.DuplicateTxnParticipantRegistration.1736586070622223-14580-0/minicluster-data/ts-0-root
uuid: "f40914ef612942d68eb28cbbfa6dac68"
format_stamp: "Formatted at 2025-01-11 09:03:13 on dist-test-slave-nvbt"
I20250111 09:03:13.442214 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.DuplicateTxnParticipantRegistration.1736586070622223-14580-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.DuplicateTxnParticipantRegistration.1736586070622223-14580-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.DuplicateTxnParticipantRegistration.1736586070622223-14580-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:13.455303 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:13.455569 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:13.456204 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:03:13.456254 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:13.456290 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:03:13.456319 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:13.461239 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:37795
I20250111 09:03:13.461305 24367 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:37795 every 8 connection(s)
I20250111 09:03:13.462219 24368 heartbeater.cc:346] Connected to a master server at 127.14.61.62:36935
I20250111 09:03:13.462297 24368 heartbeater.cc:463] Registering TS with master...
I20250111 09:03:13.462495 24368 heartbeater.cc:510] Master 127.14.61.62:36935 requested a full tablet report, sending...
I20250111 09:03:13.463070 24216 ts_manager.cc:194] Registered new tserver with Master: f40914ef612942d68eb28cbbfa6dac68 (127.14.61.1:37795)
I20250111 09:03:13.463156 14580 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.001446545s
I20250111 09:03:13.464006 24216 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33258
I20250111 09:03:13.469267 24216 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33276:
name: "txn_op_dispatcher_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
I20250111 09:03:13.476209 24327 tablet_service.cc:1467] Processing CreateTablet for tablet 635578d236e14614a1409319b284d398 (DEFAULT_TABLE table=txn_op_dispatcher_test [id=2086097982534d2bbc39751958e6862c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:13.476281 24325 tablet_service.cc:1467] Processing CreateTablet for tablet 3c86acc1bc2a44c5bb4c55c50c28764c (DEFAULT_TABLE table=txn_op_dispatcher_test [id=2086097982534d2bbc39751958e6862c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:13.476478 24327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 635578d236e14614a1409319b284d398. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:13.476606 24325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c86acc1bc2a44c5bb4c55c50c28764c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:13.478405 24378 tablet_bootstrap.cc:492] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68: Bootstrap starting.
I20250111 09:03:13.479106 24378 tablet_bootstrap.cc:654] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:13.479936 24378 tablet_bootstrap.cc:492] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68: No bootstrap required, opened a new log
I20250111 09:03:13.480006 24378 ts_tablet_manager.cc:1397] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:13.480331 24378 raft_consensus.cc:357] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40914ef612942d68eb28cbbfa6dac68" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37795 } }
I20250111 09:03:13.480409 24378 raft_consensus.cc:383] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:13.480444 24378 raft_consensus.cc:738] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f40914ef612942d68eb28cbbfa6dac68, State: Initialized, Role: FOLLOWER
I20250111 09:03:13.480566 24378 consensus_queue.cc:260] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40914ef612942d68eb28cbbfa6dac68" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37795 } }
I20250111 09:03:13.480643 24378 raft_consensus.cc:397] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:13.480677 24378 raft_consensus.cc:491] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:13.480722 24378 raft_consensus.cc:3054] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:13.481338 24378 raft_consensus.cc:513] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40914ef612942d68eb28cbbfa6dac68" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37795 } }
I20250111 09:03:13.481464 24378 leader_election.cc:304] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f40914ef612942d68eb28cbbfa6dac68; no voters:
I20250111 09:03:13.481623 24378 leader_election.cc:290] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:13.481700 24380 raft_consensus.cc:2798] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:13.481863 24380 raft_consensus.cc:695] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68 [term 1 LEADER]: Becoming Leader. State: Replica: f40914ef612942d68eb28cbbfa6dac68, State: Running, Role: LEADER
I20250111 09:03:13.481916 24378 ts_tablet_manager.cc:1428] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:13.482103 24368 heartbeater.cc:502] Master 127.14.61.62:36935 was elected leader, sending a full tablet report...
I20250111 09:03:13.482062 24380 consensus_queue.cc:237] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40914ef612942d68eb28cbbfa6dac68" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37795 } }
I20250111 09:03:13.482388 24378 tablet_bootstrap.cc:492] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68: Bootstrap starting.
I20250111 09:03:13.483297 24378 tablet_bootstrap.cc:654] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:13.483341 24216 catalog_manager.cc:5526] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68 reported cstate change: term changed from 0 to 1, leader changed from <none> to f40914ef612942d68eb28cbbfa6dac68 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "f40914ef612942d68eb28cbbfa6dac68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40914ef612942d68eb28cbbfa6dac68" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37795 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:13.484315 24378 tablet_bootstrap.cc:492] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68: No bootstrap required, opened a new log
I20250111 09:03:13.484398 24378 ts_tablet_manager.cc:1397] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:13.484791 24378 raft_consensus.cc:357] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40914ef612942d68eb28cbbfa6dac68" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37795 } }
I20250111 09:03:13.484862 24378 raft_consensus.cc:383] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:13.484884 24378 raft_consensus.cc:738] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f40914ef612942d68eb28cbbfa6dac68, State: Initialized, Role: FOLLOWER
I20250111 09:03:13.484995 24378 consensus_queue.cc:260] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40914ef612942d68eb28cbbfa6dac68" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37795 } }
I20250111 09:03:13.485141 24378 raft_consensus.cc:397] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:13.485186 24378 raft_consensus.cc:491] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:13.485234 24378 raft_consensus.cc:3054] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:13.486044 24378 raft_consensus.cc:513] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40914ef612942d68eb28cbbfa6dac68" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37795 } }
I20250111 09:03:13.486166 24378 leader_election.cc:304] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f40914ef612942d68eb28cbbfa6dac68; no voters:
I20250111 09:03:13.486259 24378 leader_election.cc:290] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:13.486325 24380 raft_consensus.cc:2798] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:13.486434 24380 raft_consensus.cc:695] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68 [term 1 LEADER]: Becoming Leader. State: Replica: f40914ef612942d68eb28cbbfa6dac68, State: Running, Role: LEADER
I20250111 09:03:13.486474 24378 ts_tablet_manager.cc:1428] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:13.486593 24380 consensus_queue.cc:237] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40914ef612942d68eb28cbbfa6dac68" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37795 } }
I20250111 09:03:13.487514 24216 catalog_manager.cc:5526] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68 reported cstate change: term changed from 0 to 1, leader changed from <none> to f40914ef612942d68eb28cbbfa6dac68 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "f40914ef612942d68eb28cbbfa6dac68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40914ef612942d68eb28cbbfa6dac68" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37795 } health_report { overall_health: HEALTHY } } }
W20250111 09:03:13.491539 24372 txn_manager_proxy_rpc.cc:150] re-attempting BeginTransaction request to one of TxnManagers
I20250111 09:03:14.436123 24216 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33280:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:14.440155 24327 tablet_service.cc:1467] Processing CreateTablet for tablet 98fb97238fe04fada9fa9baf8176a0e6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=90dcd7e77fba4881aac1900c15a4b88b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:03:14.440397 24327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98fb97238fe04fada9fa9baf8176a0e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:14.442250 24396 tablet_bootstrap.cc:492] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68: Bootstrap starting.
I20250111 09:03:14.443077 24396 tablet_bootstrap.cc:654] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:14.444021 24396 tablet_bootstrap.cc:492] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68: No bootstrap required, opened a new log
I20250111 09:03:14.444092 24396 ts_tablet_manager.cc:1397] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:14.444422 24396 raft_consensus.cc:357] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40914ef612942d68eb28cbbfa6dac68" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37795 } }
I20250111 09:03:14.444499 24396 raft_consensus.cc:383] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:14.444525 24396 raft_consensus.cc:738] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f40914ef612942d68eb28cbbfa6dac68, State: Initialized, Role: FOLLOWER
I20250111 09:03:14.444599 24396 consensus_queue.cc:260] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40914ef612942d68eb28cbbfa6dac68" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37795 } }
I20250111 09:03:14.444649 24396 raft_consensus.cc:397] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:14.444672 24396 raft_consensus.cc:491] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:14.444701 24396 raft_consensus.cc:3054] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:14.445379 24396 raft_consensus.cc:513] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40914ef612942d68eb28cbbfa6dac68" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37795 } }
I20250111 09:03:14.445508 24396 leader_election.cc:304] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f40914ef612942d68eb28cbbfa6dac68; no voters:
I20250111 09:03:14.445716 24396 leader_election.cc:290] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:14.445808 24398 raft_consensus.cc:2798] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:14.446031 24396 ts_tablet_manager.cc:1428] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:14.446080 24398 raft_consensus.cc:695] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68 [term 1 LEADER]: Becoming Leader. State: Replica: f40914ef612942d68eb28cbbfa6dac68, State: Running, Role: LEADER
I20250111 09:03:14.446274 24398 consensus_queue.cc:237] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40914ef612942d68eb28cbbfa6dac68" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37795 } }
I20250111 09:03:14.446650 24399 tablet_replica.cc:440] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f40914ef612942d68eb28cbbfa6dac68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40914ef612942d68eb28cbbfa6dac68" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37795 } } }
I20250111 09:03:14.446743 24400 tablet_replica.cc:440] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68: TxnStatusTablet state changed. Reason: New leader f40914ef612942d68eb28cbbfa6dac68. Latest consensus state: current_term: 1 leader_uuid: "f40914ef612942d68eb28cbbfa6dac68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40914ef612942d68eb28cbbfa6dac68" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37795 } } }
I20250111 09:03:14.446799 24399 tablet_replica.cc:443] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:14.446858 24400 tablet_replica.cc:443] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:14.447407 24403 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:03:14.447496 24403 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:03:14.447484 24216 catalog_manager.cc:5526] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68 reported cstate change: term changed from 0 to 1, leader changed from <none> to f40914ef612942d68eb28cbbfa6dac68 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "f40914ef612942d68eb28cbbfa6dac68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40914ef612942d68eb28cbbfa6dac68" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 37795 } health_report { overall_health: HEALTHY } } }
W20250111 09:03:15.505285 24408 meta_cache.cc:302] tablet 635578d236e14614a1409319b284d398: replica f40914ef612942d68eb28cbbfa6dac68 (127.14.61.1:37795) has failed: Timed out: Write RPC to 127.14.61.1:37795 timed out after 0.999s (SENT)
W20250111 09:03:15.505391 24408 batcher.cc:441] Timed out: Failed to write batch of 1 ops to tablet 635578d236e14614a1409319b284d398 after 1 attempt(s): Failed to write to server: f40914ef612942d68eb28cbbfa6dac68 (127.14.61.1:37795): Write RPC to 127.14.61.1:37795 timed out after 0.999s (SENT)
W20250111 09:03:16.512379 24413 tablet_replica.cc:1307] Timed out: time out prior submitting BEGIN_TXN for participant 635578d236e14614a1409319b284d398 (txn ID 0): cancelling pending write operations [suppressed 1 similar messages]
W20250111 09:03:16.512478 24413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56658: Timed out: time out prior submitting BEGIN_TXN for participant 635578d236e14614a1409319b284d398 (txn ID 0)
W20250111 09:03:16.512617 24413 rpcz_store.cc:267] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:56658 (ReqId={client: f71997a80cc84d1bb96ccd7220e47faa, seq_no=0, attempt_no=0}) took 2006 ms (client timeout 999 ms). Trace:
W20250111 09:03:16.512668 24413 rpcz_store.cc:269] 0111 09:03:14.506536 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:14.506577 (+ 41us) service_pool.cc:225] Handling call
0111 09:03:16.512606 (+2006029us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":67,"threads_started":1,"txn-participant-registration.queue_time_us":131}
I20250111 09:03:16.566059 14580 tablet_server.cc:178] TabletServer@127.14.61.1:0 shutting down...
I20250111 09:03:16.569125 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250111 09:03:16.569300 14580 tablet_replica.cc:331] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68: stopping tablet replica
I20250111 09:03:16.569401 14580 raft_consensus.cc:2238] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:16.569486 14580 raft_consensus.cc:2267] T 98fb97238fe04fada9fa9baf8176a0e6 P f40914ef612942d68eb28cbbfa6dac68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:16.569866 14580 tablet_replica.cc:331] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68: stopping tablet replica
I20250111 09:03:16.569934 14580 raft_consensus.cc:2238] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:16.570001 14580 raft_consensus.cc:2267] T 635578d236e14614a1409319b284d398 P f40914ef612942d68eb28cbbfa6dac68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:16.570432 14580 tablet_replica.cc:331] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68: stopping tablet replica
I20250111 09:03:16.570501 14580 raft_consensus.cc:2238] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:16.570569 14580 raft_consensus.cc:2267] T 3c86acc1bc2a44c5bb4c55c50c28764c P f40914ef612942d68eb28cbbfa6dac68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:16.573148 14580 tablet_server.cc:195] TabletServer@127.14.61.1:0 shutdown complete.
I20250111 09:03:16.575203 14580 master.cc:537] Master@127.14.61.62:36935 shutting down...
I20250111 09:03:16.578271 14580 raft_consensus.cc:2238] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:16.578379 14580 raft_consensus.cc:2267] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:16.578436 14580 tablet_replica.cc:331] T 00000000000000000000000000000000 P 155ea70706fa41d8809872614a81aedd: stopping tablet replica
I20250111 09:03:16.590092 14580 master.cc:559] Master@127.14.61.62:36935 shutdown complete.
[ OK ] TxnOpDispatcherITest.DuplicateTxnParticipantRegistration (3226 ms)
[ RUN ] TxnOpDispatcherITest.RollbackWriteOpPendingParticipantNotYetRegistered
I20250111 09:03:16.594798 14580 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.61.62:40385
I20250111 09:03:16.595093 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:16.597083 24429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:16.597880 24430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:16.598575 24432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:16.599206 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:16.599349 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:16.599382 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:16.599397 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586196599397 us; error 0 us; skew 500 ppm
I20250111 09:03:16.600150 14580 webserver.cc:458] Webserver started at http://127.14.61.62:38397/ using document root <none> and password file <none>
I20250111 09:03:16.600272 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:16.600312 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:16.600373 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:16.600744 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.RollbackWriteOpPendingParticipantNotYetRegistered.1736586070622223-14580-0/minicluster-data/master-0-root/instance:
uuid: "624e078989be47338f4ecad4ba6872f6"
format_stamp: "Formatted at 2025-01-11 09:03:16 on dist-test-slave-nvbt"
I20250111 09:03:16.602046 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250111 09:03:16.602789 24437 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:16.602988 14580 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250111 09:03:16.603047 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.RollbackWriteOpPendingParticipantNotYetRegistered.1736586070622223-14580-0/minicluster-data/master-0-root
uuid: "624e078989be47338f4ecad4ba6872f6"
format_stamp: "Formatted at 2025-01-11 09:03:16 on dist-test-slave-nvbt"
I20250111 09:03:16.603106 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.RollbackWriteOpPendingParticipantNotYetRegistered.1736586070622223-14580-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.RollbackWriteOpPendingParticipantNotYetRegistered.1736586070622223-14580-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.RollbackWriteOpPendingParticipantNotYetRegistered.1736586070622223-14580-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:16.617930 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:16.618188 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:16.622049 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:40385
I20250111 09:03:16.623212 24499 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:40385 every 8 connection(s)
I20250111 09:03:16.623546 24500 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:16.625054 24500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6: Bootstrap starting.
I20250111 09:03:16.625742 24500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:16.626566 24500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6: No bootstrap required, opened a new log
I20250111 09:03:16.626864 24500 raft_consensus.cc:357] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "624e078989be47338f4ecad4ba6872f6" member_type: VOTER }
I20250111 09:03:16.626933 24500 raft_consensus.cc:383] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:16.626955 24500 raft_consensus.cc:738] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 624e078989be47338f4ecad4ba6872f6, State: Initialized, Role: FOLLOWER
I20250111 09:03:16.627054 24500 consensus_queue.cc:260] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "624e078989be47338f4ecad4ba6872f6" member_type: VOTER }
I20250111 09:03:16.627120 24500 raft_consensus.cc:397] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:16.627142 24500 raft_consensus.cc:491] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:16.627171 24500 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:16.627722 24500 raft_consensus.cc:513] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "624e078989be47338f4ecad4ba6872f6" member_type: VOTER }
I20250111 09:03:16.627817 24500 leader_election.cc:304] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 624e078989be47338f4ecad4ba6872f6; no voters:
I20250111 09:03:16.627938 24500 leader_election.cc:290] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:16.628063 24503 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:16.628252 24503 raft_consensus.cc:695] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6 [term 1 LEADER]: Becoming Leader. State: Replica: 624e078989be47338f4ecad4ba6872f6, State: Running, Role: LEADER
I20250111 09:03:16.628330 24500 sys_catalog.cc:564] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6 [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:03:16.628363 24503 consensus_queue.cc:237] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "624e078989be47338f4ecad4ba6872f6" member_type: VOTER }
I20250111 09:03:16.628724 24504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "624e078989be47338f4ecad4ba6872f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "624e078989be47338f4ecad4ba6872f6" member_type: VOTER } }
I20250111 09:03:16.628762 24505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 624e078989be47338f4ecad4ba6872f6. Latest consensus state: current_term: 1 leader_uuid: "624e078989be47338f4ecad4ba6872f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "624e078989be47338f4ecad4ba6872f6" member_type: VOTER } }
I20250111 09:03:16.628882 24505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6 [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:16.629132 24504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6 [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:16.629348 24510 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:03:16.630106 24510 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:03:16.630280 14580 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250111 09:03:16.631803 24510 catalog_manager.cc:1348] Generated new cluster ID: 18817bf1dcb84043999ec21138f08872
I20250111 09:03:16.631850 24510 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:03:16.640573 24510 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:03:16.641047 24510 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:03:16.647718 24510 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6: Generated new TSK 0
I20250111 09:03:16.647857 24510 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:03:16.656934 24454 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56370:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:16.662508 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:16.664348 24527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:16.665031 24528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:16.665931 24530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:16.666714 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:16.666848 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:16.666883 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:16.666901 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586196666901 us; error 0 us; skew 500 ppm
I20250111 09:03:16.667580 14580 webserver.cc:458] Webserver started at http://127.14.61.1:36093/ using document root <none> and password file <none>
I20250111 09:03:16.667708 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:16.667749 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:16.667809 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:16.668084 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.RollbackWriteOpPendingParticipantNotYetRegistered.1736586070622223-14580-0/minicluster-data/ts-0-root/instance:
uuid: "c41dee24bf5549c09e49779adef97b90"
format_stamp: "Formatted at 2025-01-11 09:03:16 on dist-test-slave-nvbt"
I20250111 09:03:16.669261 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250111 09:03:16.669972 24535 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:16.670161 14580 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250111 09:03:16.670218 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.RollbackWriteOpPendingParticipantNotYetRegistered.1736586070622223-14580-0/minicluster-data/ts-0-root
uuid: "c41dee24bf5549c09e49779adef97b90"
format_stamp: "Formatted at 2025-01-11 09:03:16 on dist-test-slave-nvbt"
I20250111 09:03:16.670284 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.RollbackWriteOpPendingParticipantNotYetRegistered.1736586070622223-14580-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.RollbackWriteOpPendingParticipantNotYetRegistered.1736586070622223-14580-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.RollbackWriteOpPendingParticipantNotYetRegistered.1736586070622223-14580-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:16.687672 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:16.687912 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:16.688519 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:03:16.688565 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:16.688655 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:03:16.688716 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:16.693198 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:44069
I20250111 09:03:16.693226 24605 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:44069 every 8 connection(s)
I20250111 09:03:16.694245 24606 heartbeater.cc:346] Connected to a master server at 127.14.61.62:40385
I20250111 09:03:16.694317 24606 heartbeater.cc:463] Registering TS with master...
I20250111 09:03:16.694525 24606 heartbeater.cc:510] Master 127.14.61.62:40385 requested a full tablet report, sending...
I20250111 09:03:16.695021 24454 ts_manager.cc:194] Registered new tserver with Master: c41dee24bf5549c09e49779adef97b90 (127.14.61.1:44069)
I20250111 09:03:16.695668 24454 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56382
I20250111 09:03:16.695976 14580 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002254586s
I20250111 09:03:16.701187 24454 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56402:
name: "txn_op_dispatcher_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
I20250111 09:03:16.707787 24565 tablet_service.cc:1467] Processing CreateTablet for tablet c2720b5b62d7474599abc515365b00d2 (DEFAULT_TABLE table=txn_op_dispatcher_test [id=2fa97ea6a53a4b449cca951866b4bb91]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:16.707864 24564 tablet_service.cc:1467] Processing CreateTablet for tablet 34287ea803c34cc693ed092f36592943 (DEFAULT_TABLE table=txn_op_dispatcher_test [id=2fa97ea6a53a4b449cca951866b4bb91]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:16.708019 24565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2720b5b62d7474599abc515365b00d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:16.708180 24564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34287ea803c34cc693ed092f36592943. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:16.709714 24616 tablet_bootstrap.cc:492] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90: Bootstrap starting.
I20250111 09:03:16.710592 24616 tablet_bootstrap.cc:654] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:16.711386 24616 tablet_bootstrap.cc:492] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90: No bootstrap required, opened a new log
I20250111 09:03:16.711450 24616 ts_tablet_manager.cc:1397] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:16.711735 24616 raft_consensus.cc:357] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41dee24bf5549c09e49779adef97b90" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44069 } }
I20250111 09:03:16.711804 24616 raft_consensus.cc:383] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:16.711829 24616 raft_consensus.cc:738] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c41dee24bf5549c09e49779adef97b90, State: Initialized, Role: FOLLOWER
I20250111 09:03:16.711908 24616 consensus_queue.cc:260] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41dee24bf5549c09e49779adef97b90" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44069 } }
I20250111 09:03:16.711958 24616 raft_consensus.cc:397] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:16.711982 24616 raft_consensus.cc:491] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:16.712013 24616 raft_consensus.cc:3054] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:16.712558 24616 raft_consensus.cc:513] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41dee24bf5549c09e49779adef97b90" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44069 } }
I20250111 09:03:16.712651 24616 leader_election.cc:304] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c41dee24bf5549c09e49779adef97b90; no voters:
I20250111 09:03:16.712779 24616 leader_election.cc:290] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:16.712862 24618 raft_consensus.cc:2798] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:16.713035 24618 raft_consensus.cc:695] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90 [term 1 LEADER]: Becoming Leader. State: Replica: c41dee24bf5549c09e49779adef97b90, State: Running, Role: LEADER
I20250111 09:03:16.713048 24616 ts_tablet_manager.cc:1428] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:16.713064 24606 heartbeater.cc:502] Master 127.14.61.62:40385 was elected leader, sending a full tablet report...
I20250111 09:03:16.713312 24616 tablet_bootstrap.cc:492] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90: Bootstrap starting.
I20250111 09:03:16.714093 24616 tablet_bootstrap.cc:654] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:16.714583 24618 consensus_queue.cc:237] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41dee24bf5549c09e49779adef97b90" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44069 } }
I20250111 09:03:16.714934 24616 tablet_bootstrap.cc:492] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90: No bootstrap required, opened a new log
I20250111 09:03:16.715003 24616 ts_tablet_manager.cc:1397] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:16.715391 24616 raft_consensus.cc:357] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41dee24bf5549c09e49779adef97b90" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44069 } }
I20250111 09:03:16.715480 24616 raft_consensus.cc:383] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:16.715512 24616 raft_consensus.cc:738] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c41dee24bf5549c09e49779adef97b90, State: Initialized, Role: FOLLOWER
I20250111 09:03:16.715603 24616 consensus_queue.cc:260] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41dee24bf5549c09e49779adef97b90" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44069 } }
I20250111 09:03:16.715649 24616 raft_consensus.cc:397] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:16.715668 24616 raft_consensus.cc:491] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:16.715692 24616 raft_consensus.cc:3054] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:16.715859 24454 catalog_manager.cc:5526] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90 reported cstate change: term changed from 0 to 1, leader changed from <none> to c41dee24bf5549c09e49779adef97b90 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "c41dee24bf5549c09e49779adef97b90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41dee24bf5549c09e49779adef97b90" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44069 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:16.716213 24616 raft_consensus.cc:513] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41dee24bf5549c09e49779adef97b90" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44069 } }
I20250111 09:03:16.716288 24616 leader_election.cc:304] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c41dee24bf5549c09e49779adef97b90; no voters:
I20250111 09:03:16.716346 24616 leader_election.cc:290] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:16.716398 24618 raft_consensus.cc:2798] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:16.716532 24616 ts_tablet_manager.cc:1428] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:16.716688 24618 raft_consensus.cc:695] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90 [term 1 LEADER]: Becoming Leader. State: Replica: c41dee24bf5549c09e49779adef97b90, State: Running, Role: LEADER
I20250111 09:03:16.716814 24618 consensus_queue.cc:237] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41dee24bf5549c09e49779adef97b90" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44069 } }
I20250111 09:03:16.718225 24454 catalog_manager.cc:5526] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90 reported cstate change: term changed from 0 to 1, leader changed from <none> to c41dee24bf5549c09e49779adef97b90 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "c41dee24bf5549c09e49779adef97b90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41dee24bf5549c09e49779adef97b90" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44069 } health_report { overall_health: HEALTHY } } }
W20250111 09:03:16.723224 24610 txn_manager_proxy_rpc.cc:150] re-attempting BeginTransaction request to one of TxnManagers
I20250111 09:03:17.664153 24454 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56404:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:17.668412 24565 tablet_service.cc:1467] Processing CreateTablet for tablet 81a9eef99ddd47c8821591ee0a4db5f3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cabfcb5004da4e39b59cdbc86a64c74c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:03:17.668654 24565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81a9eef99ddd47c8821591ee0a4db5f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:17.670370 24634 tablet_bootstrap.cc:492] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90: Bootstrap starting.
I20250111 09:03:17.671103 24634 tablet_bootstrap.cc:654] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:17.672062 24634 tablet_bootstrap.cc:492] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90: No bootstrap required, opened a new log
I20250111 09:03:17.672138 24634 ts_tablet_manager.cc:1397] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:17.672454 24634 raft_consensus.cc:357] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41dee24bf5549c09e49779adef97b90" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44069 } }
I20250111 09:03:17.672529 24634 raft_consensus.cc:383] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:17.672560 24634 raft_consensus.cc:738] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c41dee24bf5549c09e49779adef97b90, State: Initialized, Role: FOLLOWER
I20250111 09:03:17.672667 24634 consensus_queue.cc:260] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41dee24bf5549c09e49779adef97b90" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44069 } }
I20250111 09:03:17.672729 24634 raft_consensus.cc:397] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:17.672765 24634 raft_consensus.cc:491] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:17.672809 24634 raft_consensus.cc:3054] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:17.673506 24634 raft_consensus.cc:513] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41dee24bf5549c09e49779adef97b90" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44069 } }
I20250111 09:03:17.673625 24634 leader_election.cc:304] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c41dee24bf5549c09e49779adef97b90; no voters:
I20250111 09:03:17.673796 24634 leader_election.cc:290] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:17.673882 24636 raft_consensus.cc:2798] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:17.674046 24636 raft_consensus.cc:695] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90 [term 1 LEADER]: Becoming Leader. State: Replica: c41dee24bf5549c09e49779adef97b90, State: Running, Role: LEADER
I20250111 09:03:17.674116 24634 ts_tablet_manager.cc:1428] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:17.674165 24636 consensus_queue.cc:237] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41dee24bf5549c09e49779adef97b90" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44069 } }
I20250111 09:03:17.674630 24638 tablet_replica.cc:440] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c41dee24bf5549c09e49779adef97b90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41dee24bf5549c09e49779adef97b90" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44069 } } }
I20250111 09:03:17.674726 24638 tablet_replica.cc:443] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:17.674852 24637 tablet_replica.cc:440] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90: TxnStatusTablet state changed. Reason: New leader c41dee24bf5549c09e49779adef97b90. Latest consensus state: current_term: 1 leader_uuid: "c41dee24bf5549c09e49779adef97b90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41dee24bf5549c09e49779adef97b90" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44069 } } }
I20250111 09:03:17.675016 24637 tablet_replica.cc:443] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:17.675037 24640 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:03:17.675379 24640 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:03:17.675402 24454 catalog_manager.cc:5526] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90 reported cstate change: term changed from 0 to 1, leader changed from <none> to c41dee24bf5549c09e49779adef97b90 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "c41dee24bf5549c09e49779adef97b90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41dee24bf5549c09e49779adef97b90" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 44069 } health_report { overall_health: HEALTHY } } }
W20250111 09:03:20.120909 24648 tablet_replica.cc:1307] Illegal state: Failed to write to server: c41dee24bf5549c09e49779adef97b90 (127.14.61.1:44069): transaction ID 0 not open: ABORTED: cancelling pending write operations
W20250111 09:03:20.121031 24648 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:49514: Illegal state: Failed to write to server: c41dee24bf5549c09e49779adef97b90 (127.14.61.1:44069): transaction ID 0 not open: ABORTED
I20250111 09:03:20.121166 24648 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:49514 (ReqId={client: dd2ad38bd39c45068a2177218e038d10, seq_no=0, attempt_no=0}) took 2006 ms. Trace:
I20250111 09:03:20.121206 24648 rpcz_store.cc:276] 0111 09:03:18.114439 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:18.114503 (+ 64us) service_pool.cc:225] Handling call
0111 09:03:20.121156 (+2006653us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":70,"threads_started":1,"txn-participant-registration.queue_time_us":140}
W20250111 09:03:20.121567 24609 batcher.cc:441] Illegal state: Failed to write batch of 1 ops to tablet c2720b5b62d7474599abc515365b00d2 after 1 attempt(s): Failed to write to server: c41dee24bf5549c09e49779adef97b90 (127.14.61.1:44069): Failed to write to server: c41dee24bf5549c09e49779adef97b90 (127.14.61.1:44069): transaction ID 0 not open: ABORTED
I20250111 09:03:20.126829 14580 tablet_server.cc:178] TabletServer@127.14.61.1:0 shutting down...
I20250111 09:03:20.129770 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250111 09:03:20.129947 14580 tablet_replica.cc:331] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90: stopping tablet replica
I20250111 09:03:20.130054 14580 raft_consensus.cc:2238] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:20.130172 14580 raft_consensus.cc:2267] T 81a9eef99ddd47c8821591ee0a4db5f3 P c41dee24bf5549c09e49779adef97b90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:20.130537 14580 tablet_replica.cc:331] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90: stopping tablet replica
I20250111 09:03:20.130623 14580 raft_consensus.cc:2238] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:20.130709 14580 raft_consensus.cc:2267] T c2720b5b62d7474599abc515365b00d2 P c41dee24bf5549c09e49779adef97b90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:20.131002 14580 tablet_replica.cc:331] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90: stopping tablet replica
I20250111 09:03:20.131085 14580 raft_consensus.cc:2238] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:20.131176 14580 raft_consensus.cc:2267] T 34287ea803c34cc693ed092f36592943 P c41dee24bf5549c09e49779adef97b90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:20.143447 14580 tablet_server.cc:195] TabletServer@127.14.61.1:0 shutdown complete.
I20250111 09:03:20.145653 14580 master.cc:537] Master@127.14.61.62:40385 shutting down...
I20250111 09:03:20.149175 14580 raft_consensus.cc:2238] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:20.149284 14580 raft_consensus.cc:2267] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:20.149341 14580 tablet_replica.cc:331] T 00000000000000000000000000000000 P 624e078989be47338f4ecad4ba6872f6: stopping tablet replica
I20250111 09:03:20.160962 14580 master.cc:559] Master@127.14.61.62:40385 shutdown complete.
[ OK ] TxnOpDispatcherITest.RollbackWriteOpPendingParticipantNotYetRegistered (3571 ms)
[ RUN ] TxnOpDispatcherITest.RollbackWriteOpPendingParticipantRegistered
I20250111 09:03:20.166069 14580 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.61.62:44447
I20250111 09:03:20.166399 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:20.168767 24658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:20.169584 24660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:20.170089 24657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:20.170692 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:20.170811 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:20.170858 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:20.170871 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586200170871 us; error 0 us; skew 500 ppm
I20250111 09:03:20.180198 14580 webserver.cc:458] Webserver started at http://127.14.61.62:38811/ using document root <none> and password file <none>
I20250111 09:03:20.180328 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:20.180366 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:20.180430 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:20.180738 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.RollbackWriteOpPendingParticipantRegistered.1736586070622223-14580-0/minicluster-data/master-0-root/instance:
uuid: "6bc78e18a0c54598a1788b7babe7aba8"
format_stamp: "Formatted at 2025-01-11 09:03:20 on dist-test-slave-nvbt"
I20250111 09:03:20.182025 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250111 09:03:20.182773 24665 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:20.182977 14580 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250111 09:03:20.183036 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.RollbackWriteOpPendingParticipantRegistered.1736586070622223-14580-0/minicluster-data/master-0-root
uuid: "6bc78e18a0c54598a1788b7babe7aba8"
format_stamp: "Formatted at 2025-01-11 09:03:20 on dist-test-slave-nvbt"
I20250111 09:03:20.183095 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.RollbackWriteOpPendingParticipantRegistered.1736586070622223-14580-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.RollbackWriteOpPendingParticipantRegistered.1736586070622223-14580-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.RollbackWriteOpPendingParticipantRegistered.1736586070622223-14580-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:20.194494 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:20.194753 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:20.198349 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:44447
I20250111 09:03:20.199136 24727 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:44447 every 8 connection(s)
I20250111 09:03:20.199477 24728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:20.200862 24728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8: Bootstrap starting.
I20250111 09:03:20.201530 24728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:20.202273 24728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8: No bootstrap required, opened a new log
I20250111 09:03:20.202574 24728 raft_consensus.cc:357] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc78e18a0c54598a1788b7babe7aba8" member_type: VOTER }
I20250111 09:03:20.202639 24728 raft_consensus.cc:383] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:20.202672 24728 raft_consensus.cc:738] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bc78e18a0c54598a1788b7babe7aba8, State: Initialized, Role: FOLLOWER
I20250111 09:03:20.202785 24728 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc78e18a0c54598a1788b7babe7aba8" member_type: VOTER }
I20250111 09:03:20.202841 24728 raft_consensus.cc:397] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:20.202875 24728 raft_consensus.cc:491] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:20.202920 24728 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:20.203466 24728 raft_consensus.cc:513] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc78e18a0c54598a1788b7babe7aba8" member_type: VOTER }
I20250111 09:03:20.203591 24728 leader_election.cc:304] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bc78e18a0c54598a1788b7babe7aba8; no voters:
I20250111 09:03:20.203763 24728 leader_election.cc:290] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:20.203851 24731 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:20.204032 24731 raft_consensus.cc:695] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8 [term 1 LEADER]: Becoming Leader. State: Replica: 6bc78e18a0c54598a1788b7babe7aba8, State: Running, Role: LEADER
I20250111 09:03:20.204146 24728 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8 [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:03:20.204167 24731 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc78e18a0c54598a1788b7babe7aba8" member_type: VOTER }
I20250111 09:03:20.204676 24732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6bc78e18a0c54598a1788b7babe7aba8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc78e18a0c54598a1788b7babe7aba8" member_type: VOTER } }
I20250111 09:03:20.204704 24733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bc78e18a0c54598a1788b7babe7aba8. Latest consensus state: current_term: 1 leader_uuid: "6bc78e18a0c54598a1788b7babe7aba8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc78e18a0c54598a1788b7babe7aba8" member_type: VOTER } }
I20250111 09:03:20.204847 24733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8 [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:20.205077 24732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8 [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:20.205329 24737 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:03:20.206080 24737 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:03:20.206265 14580 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250111 09:03:20.207854 24737 catalog_manager.cc:1348] Generated new cluster ID: 933b08bfaced4e33a7edbd5e9e5973fd
I20250111 09:03:20.207906 24737 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:03:20.228588 24737 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:03:20.229084 24737 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:03:20.238793 24737 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8: Generated new TSK 0
I20250111 09:03:20.238956 24737 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:03:20.246955 24682 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37150:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:20.270447 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:20.272590 24756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:20.273324 24755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:20.273885 24758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:20.274583 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:20.274721 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:20.274771 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:20.274786 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586200274786 us; error 0 us; skew 500 ppm
I20250111 09:03:20.275626 14580 webserver.cc:458] Webserver started at http://127.14.61.1:35173/ using document root <none> and password file <none>
I20250111 09:03:20.275761 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:20.275817 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:20.275882 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:20.276176 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.RollbackWriteOpPendingParticipantRegistered.1736586070622223-14580-0/minicluster-data/ts-0-root/instance:
uuid: "1318e6c8f277427db2f473d8f38a5fb9"
format_stamp: "Formatted at 2025-01-11 09:03:20 on dist-test-slave-nvbt"
I20250111 09:03:20.277388 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250111 09:03:20.278128 24763 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:20.278323 14580 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250111 09:03:20.278383 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.RollbackWriteOpPendingParticipantRegistered.1736586070622223-14580-0/minicluster-data/ts-0-root
uuid: "1318e6c8f277427db2f473d8f38a5fb9"
format_stamp: "Formatted at 2025-01-11 09:03:20 on dist-test-slave-nvbt"
I20250111 09:03:20.278439 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.RollbackWriteOpPendingParticipantRegistered.1736586070622223-14580-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.RollbackWriteOpPendingParticipantRegistered.1736586070622223-14580-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.RollbackWriteOpPendingParticipantRegistered.1736586070622223-14580-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:20.285768 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:20.286028 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:20.286685 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:03:20.286737 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:20.286775 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:03:20.286845 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:20.291622 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:35141
I20250111 09:03:20.291635 24833 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:35141 every 8 connection(s)
I20250111 09:03:20.292506 24834 heartbeater.cc:346] Connected to a master server at 127.14.61.62:44447
I20250111 09:03:20.292569 24834 heartbeater.cc:463] Registering TS with master...
I20250111 09:03:20.292716 24834 heartbeater.cc:510] Master 127.14.61.62:44447 requested a full tablet report, sending...
I20250111 09:03:20.293368 24682 ts_manager.cc:194] Registered new tserver with Master: 1318e6c8f277427db2f473d8f38a5fb9 (127.14.61.1:35141)
I20250111 09:03:20.294092 24682 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37154
I20250111 09:03:20.294242 14580 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002228876s
I20250111 09:03:20.299664 24682 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37178:
name: "txn_op_dispatcher_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
I20250111 09:03:20.306661 24790 tablet_service.cc:1467] Processing CreateTablet for tablet bb8e32d5c554478fa17025e3e0d58366 (DEFAULT_TABLE table=txn_op_dispatcher_test [id=e8592fef8a6346428b59691dc7e64a2a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:20.306710 24793 tablet_service.cc:1467] Processing CreateTablet for tablet 719a0987e2c34754af84f0e2bd8baad8 (DEFAULT_TABLE table=txn_op_dispatcher_test [id=e8592fef8a6346428b59691dc7e64a2a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:20.306906 24790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb8e32d5c554478fa17025e3e0d58366. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:20.307037 24793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 719a0987e2c34754af84f0e2bd8baad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:20.308630 24844 tablet_bootstrap.cc:492] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9: Bootstrap starting.
I20250111 09:03:20.309446 24844 tablet_bootstrap.cc:654] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:20.310269 24844 tablet_bootstrap.cc:492] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9: No bootstrap required, opened a new log
I20250111 09:03:20.310331 24844 ts_tablet_manager.cc:1397] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:20.310647 24844 raft_consensus.cc:357] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1318e6c8f277427db2f473d8f38a5fb9" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35141 } }
I20250111 09:03:20.310717 24844 raft_consensus.cc:383] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:20.310748 24844 raft_consensus.cc:738] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1318e6c8f277427db2f473d8f38a5fb9, State: Initialized, Role: FOLLOWER
I20250111 09:03:20.310875 24844 consensus_queue.cc:260] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1318e6c8f277427db2f473d8f38a5fb9" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35141 } }
I20250111 09:03:20.310935 24844 raft_consensus.cc:397] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:20.310985 24844 raft_consensus.cc:491] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:20.311048 24844 raft_consensus.cc:3054] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:20.311772 24844 raft_consensus.cc:513] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1318e6c8f277427db2f473d8f38a5fb9" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35141 } }
I20250111 09:03:20.311894 24844 leader_election.cc:304] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1318e6c8f277427db2f473d8f38a5fb9; no voters:
I20250111 09:03:20.312062 24844 leader_election.cc:290] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:20.312150 24846 raft_consensus.cc:2798] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:20.312340 24846 raft_consensus.cc:695] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 1318e6c8f277427db2f473d8f38a5fb9, State: Running, Role: LEADER
I20250111 09:03:20.312355 24844 ts_tablet_manager.cc:1428] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:20.312479 24834 heartbeater.cc:502] Master 127.14.61.62:44447 was elected leader, sending a full tablet report...
I20250111 09:03:20.312494 24846 consensus_queue.cc:237] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1318e6c8f277427db2f473d8f38a5fb9" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35141 } }
I20250111 09:03:20.312583 24844 tablet_bootstrap.cc:492] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9: Bootstrap starting.
I20250111 09:03:20.313410 24844 tablet_bootstrap.cc:654] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:20.313741 24682 catalog_manager.cc:5526] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1318e6c8f277427db2f473d8f38a5fb9 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "1318e6c8f277427db2f473d8f38a5fb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1318e6c8f277427db2f473d8f38a5fb9" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35141 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:20.314483 24844 tablet_bootstrap.cc:492] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9: No bootstrap required, opened a new log
I20250111 09:03:20.314569 24844 ts_tablet_manager.cc:1397] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:20.314970 24844 raft_consensus.cc:357] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1318e6c8f277427db2f473d8f38a5fb9" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35141 } }
I20250111 09:03:20.315040 24844 raft_consensus.cc:383] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:20.315069 24844 raft_consensus.cc:738] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1318e6c8f277427db2f473d8f38a5fb9, State: Initialized, Role: FOLLOWER
I20250111 09:03:20.315166 24844 consensus_queue.cc:260] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1318e6c8f277427db2f473d8f38a5fb9" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35141 } }
I20250111 09:03:20.315263 24844 raft_consensus.cc:397] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:20.315300 24844 raft_consensus.cc:491] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:20.315346 24844 raft_consensus.cc:3054] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:20.316071 24844 raft_consensus.cc:513] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1318e6c8f277427db2f473d8f38a5fb9" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35141 } }
I20250111 09:03:20.316171 24844 leader_election.cc:304] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1318e6c8f277427db2f473d8f38a5fb9; no voters:
I20250111 09:03:20.316249 24844 leader_election.cc:290] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:20.316285 24846 raft_consensus.cc:2798] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:20.316378 24846 raft_consensus.cc:695] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 1318e6c8f277427db2f473d8f38a5fb9, State: Running, Role: LEADER
I20250111 09:03:20.316468 24844 ts_tablet_manager.cc:1428] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:20.316540 24846 consensus_queue.cc:237] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1318e6c8f277427db2f473d8f38a5fb9" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35141 } }
I20250111 09:03:20.317564 24682 catalog_manager.cc:5526] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1318e6c8f277427db2f473d8f38a5fb9 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "1318e6c8f277427db2f473d8f38a5fb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1318e6c8f277427db2f473d8f38a5fb9" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35141 } health_report { overall_health: HEALTHY } } }
W20250111 09:03:20.321859 24838 txn_manager_proxy_rpc.cc:150] re-attempting BeginTransaction request to one of TxnManagers
I20250111 09:03:21.254063 24682 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37184:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:21.258283 24790 tablet_service.cc:1467] Processing CreateTablet for tablet 70bdd1fec0544b5db317327874eb2882 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=71e25e7a69ee45b994a542fa2e8daddd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:03:21.258544 24790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70bdd1fec0544b5db317327874eb2882. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:21.260252 24862 tablet_bootstrap.cc:492] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9: Bootstrap starting.
I20250111 09:03:21.260938 24862 tablet_bootstrap.cc:654] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:21.261819 24862 tablet_bootstrap.cc:492] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9: No bootstrap required, opened a new log
I20250111 09:03:21.261883 24862 ts_tablet_manager.cc:1397] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:21.262143 24862 raft_consensus.cc:357] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1318e6c8f277427db2f473d8f38a5fb9" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35141 } }
I20250111 09:03:21.262212 24862 raft_consensus.cc:383] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:21.262244 24862 raft_consensus.cc:738] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1318e6c8f277427db2f473d8f38a5fb9, State: Initialized, Role: FOLLOWER
I20250111 09:03:21.262332 24862 consensus_queue.cc:260] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1318e6c8f277427db2f473d8f38a5fb9" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35141 } }
I20250111 09:03:21.262399 24862 raft_consensus.cc:397] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:21.262424 24862 raft_consensus.cc:491] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:21.262454 24862 raft_consensus.cc:3054] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:21.263034 24862 raft_consensus.cc:513] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1318e6c8f277427db2f473d8f38a5fb9" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35141 } }
I20250111 09:03:21.263129 24862 leader_election.cc:304] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1318e6c8f277427db2f473d8f38a5fb9; no voters:
I20250111 09:03:21.263253 24862 leader_election.cc:290] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:21.263336 24864 raft_consensus.cc:2798] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:21.263532 24862 ts_tablet_manager.cc:1428] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:21.263556 24864 raft_consensus.cc:695] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 1318e6c8f277427db2f473d8f38a5fb9, State: Running, Role: LEADER
I20250111 09:03:21.263835 24864 consensus_queue.cc:237] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1318e6c8f277427db2f473d8f38a5fb9" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35141 } }
I20250111 09:03:21.264227 24866 tablet_replica.cc:440] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9: TxnStatusTablet state changed. Reason: New leader 1318e6c8f277427db2f473d8f38a5fb9. Latest consensus state: current_term: 1 leader_uuid: "1318e6c8f277427db2f473d8f38a5fb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1318e6c8f277427db2f473d8f38a5fb9" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35141 } } }
I20250111 09:03:21.264386 24866 tablet_replica.cc:443] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:21.264372 24865 tablet_replica.cc:440] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1318e6c8f277427db2f473d8f38a5fb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1318e6c8f277427db2f473d8f38a5fb9" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35141 } } }
I20250111 09:03:21.264524 24865 tablet_replica.cc:443] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:21.264827 24868 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:03:21.264941 24868 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:03:21.265131 24682 catalog_manager.cc:5526] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1318e6c8f277427db2f473d8f38a5fb9 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "1318e6c8f277427db2f473d8f38a5fb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1318e6c8f277427db2f473d8f38a5fb9" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 35141 } health_report { overall_health: HEALTHY } } }
W20250111 09:03:22.403334 24879 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations
W20250111 09:03:22.403415 24879 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:52284: Aborted: operation has been aborted
I20250111 09:03:22.403506 24879 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:52284 (ReqId={client: 8689e7f59050498d96d5082cd28ba952, seq_no=0, attempt_no=0}) took 1001 ms. Trace:
I20250111 09:03:22.403537 24879 rpcz_store.cc:276] 0111 09:03:21.402493 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:21.402537 (+ 44us) service_pool.cc:225] Handling call
0111 09:03:22.403500 (+1000963us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":86,"threads_started":1,"txn-participant-registration.queue_time_us":165}
W20250111 09:03:22.403815 24839 batcher.cc:441] Aborted: Failed to write batch of 1 ops to tablet bb8e32d5c554478fa17025e3e0d58366 after 1 attempt(s): Failed to write to server: 1318e6c8f277427db2f473d8f38a5fb9 (127.14.61.1:35141): operation has been aborted
I20250111 09:03:22.410915 14580 tablet_server.cc:178] TabletServer@127.14.61.1:0 shutting down...
I20250111 09:03:22.413846 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
W20250111 09:03:23.409790 24884 tablet_replica.cc:1307] Illegal state: Transaction metadata for transaction 0 already exists: cancelling pending write operations
I20250111 09:03:23.409826 14580 tablet_replica.cc:331] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9: stopping tablet replica
I20250111 09:03:23.409991 14580 raft_consensus.cc:2238] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:23.410125 14580 raft_consensus.cc:2267] T 70bdd1fec0544b5db317327874eb2882 P 1318e6c8f277427db2f473d8f38a5fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:23.410532 14580 tablet_replica.cc:331] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9: stopping tablet replica
I20250111 09:03:23.410611 14580 raft_consensus.cc:2238] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:23.410706 14580 raft_consensus.cc:2267] T bb8e32d5c554478fa17025e3e0d58366 P 1318e6c8f277427db2f473d8f38a5fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:23.410976 14580 tablet_replica.cc:331] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9: stopping tablet replica
I20250111 09:03:23.411039 14580 raft_consensus.cc:2238] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:23.411125 14580 raft_consensus.cc:2267] T 719a0987e2c34754af84f0e2bd8baad8 P 1318e6c8f277427db2f473d8f38a5fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:23.423697 14580 tablet_server.cc:195] TabletServer@127.14.61.1:0 shutdown complete.
I20250111 09:03:23.426039 14580 master.cc:537] Master@127.14.61.62:44447 shutting down...
I20250111 09:03:23.429234 14580 raft_consensus.cc:2238] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:23.429327 14580 raft_consensus.cc:2267] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:23.429366 14580 tablet_replica.cc:331] T 00000000000000000000000000000000 P 6bc78e18a0c54598a1788b7babe7aba8: stopping tablet replica
I20250111 09:03:23.441329 14580 master.cc:559] Master@127.14.61.62:44447 shutdown complete.
[ OK ] TxnOpDispatcherITest.RollbackWriteOpPendingParticipantRegistered (3279 ms)
[ RUN ] TxnOpDispatcherITest.TxnWriteWhileReplicaDeleted
I20250111 09:03:23.445943 14580 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.61.62:38671
I20250111 09:03:23.446204 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:23.448127 24885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:23.448823 14580 server_base.cc:1034] running on GCE node
W20250111 09:03:23.449664 24888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:23.450405 24886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:23.450595 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:23.450639 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:23.450660 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586203450659 us; error 0 us; skew 500 ppm
I20250111 09:03:23.451448 14580 webserver.cc:458] Webserver started at http://127.14.61.62:42175/ using document root <none> and password file <none>
I20250111 09:03:23.451581 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:23.451629 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:23.451702 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:23.452030 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhileReplicaDeleted.1736586070622223-14580-0/minicluster-data/master-0-root/instance:
uuid: "ef538da7a18b4fadb320f5d722d799bc"
format_stamp: "Formatted at 2025-01-11 09:03:23 on dist-test-slave-nvbt"
I20250111 09:03:23.453343 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250111 09:03:23.454164 24893 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:23.454437 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:23.454504 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhileReplicaDeleted.1736586070622223-14580-0/minicluster-data/master-0-root
uuid: "ef538da7a18b4fadb320f5d722d799bc"
format_stamp: "Formatted at 2025-01-11 09:03:23 on dist-test-slave-nvbt"
I20250111 09:03:23.454557 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhileReplicaDeleted.1736586070622223-14580-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhileReplicaDeleted.1736586070622223-14580-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhileReplicaDeleted.1736586070622223-14580-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:23.475189 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:23.475452 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:23.479449 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:38671
I20250111 09:03:23.480242 24955 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:38671 every 8 connection(s)
I20250111 09:03:23.480612 24956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:23.482147 24956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc: Bootstrap starting.
I20250111 09:03:23.482841 24956 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:23.483635 24956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc: No bootstrap required, opened a new log
I20250111 09:03:23.483973 24956 raft_consensus.cc:357] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef538da7a18b4fadb320f5d722d799bc" member_type: VOTER }
I20250111 09:03:23.484050 24956 raft_consensus.cc:383] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:23.484086 24956 raft_consensus.cc:738] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef538da7a18b4fadb320f5d722d799bc, State: Initialized, Role: FOLLOWER
I20250111 09:03:23.484201 24956 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef538da7a18b4fadb320f5d722d799bc" member_type: VOTER }
I20250111 09:03:23.484267 24956 raft_consensus.cc:397] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:23.484311 24956 raft_consensus.cc:491] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:23.484359 24956 raft_consensus.cc:3054] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:23.485245 24956 raft_consensus.cc:513] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef538da7a18b4fadb320f5d722d799bc" member_type: VOTER }
I20250111 09:03:23.485376 24956 leader_election.cc:304] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef538da7a18b4fadb320f5d722d799bc; no voters:
I20250111 09:03:23.485540 24956 leader_election.cc:290] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:23.485630 24959 raft_consensus.cc:2798] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:23.485810 24959 raft_consensus.cc:695] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc [term 1 LEADER]: Becoming Leader. State: Replica: ef538da7a18b4fadb320f5d722d799bc, State: Running, Role: LEADER
I20250111 09:03:23.485947 24956 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:03:23.485955 24959 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef538da7a18b4fadb320f5d722d799bc" member_type: VOTER }
I20250111 09:03:23.486387 24960 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef538da7a18b4fadb320f5d722d799bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef538da7a18b4fadb320f5d722d799bc" member_type: VOTER } }
I20250111 09:03:23.486426 24961 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef538da7a18b4fadb320f5d722d799bc. Latest consensus state: current_term: 1 leader_uuid: "ef538da7a18b4fadb320f5d722d799bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef538da7a18b4fadb320f5d722d799bc" member_type: VOTER } }
I20250111 09:03:23.486570 24961 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:23.486811 24960 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:23.486984 24966 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:03:23.487694 24966 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:03:23.487912 14580 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250111 09:03:23.489315 24966 catalog_manager.cc:1348] Generated new cluster ID: 3c508d400ebe44a09e03de6179f259ee
I20250111 09:03:23.489364 24966 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:03:23.500375 24966 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:03:23.500792 24966 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:03:23.510056 24966 catalog_manager.cc:5899] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc: Generated new TSK 0
I20250111 09:03:23.510186 24966 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:03:23.518261 24910 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46896:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:23.519909 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:23.521888 24983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:23.522760 24984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:23.523422 24986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:23.524070 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:23.524195 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:23.524231 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:23.524246 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586203524247 us; error 0 us; skew 500 ppm
I20250111 09:03:23.525084 14580 webserver.cc:458] Webserver started at http://127.14.61.1:37315/ using document root <none> and password file <none>
I20250111 09:03:23.525202 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:23.525239 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:23.525285 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:23.525585 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhileReplicaDeleted.1736586070622223-14580-0/minicluster-data/ts-0-root/instance:
uuid: "57e08ebe49d94c4eb09595ec02c012ab"
format_stamp: "Formatted at 2025-01-11 09:03:23 on dist-test-slave-nvbt"
I20250111 09:03:23.526887 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250111 09:03:23.527650 24991 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:23.527915 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20250111 09:03:23.527974 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhileReplicaDeleted.1736586070622223-14580-0/minicluster-data/ts-0-root
uuid: "57e08ebe49d94c4eb09595ec02c012ab"
format_stamp: "Formatted at 2025-01-11 09:03:23 on dist-test-slave-nvbt"
I20250111 09:03:23.528041 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhileReplicaDeleted.1736586070622223-14580-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhileReplicaDeleted.1736586070622223-14580-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhileReplicaDeleted.1736586070622223-14580-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:23.551268 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:23.551515 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:23.552124 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:03:23.552188 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:23.552254 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:03:23.552306 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:23.557248 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:38013
I20250111 09:03:23.557838 25061 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:38013 every 8 connection(s)
I20250111 09:03:23.558140 25062 heartbeater.cc:346] Connected to a master server at 127.14.61.62:38671
I20250111 09:03:23.558223 25062 heartbeater.cc:463] Registering TS with master...
I20250111 09:03:23.558410 25062 heartbeater.cc:510] Master 127.14.61.62:38671 requested a full tablet report, sending...
I20250111 09:03:23.558954 24910 ts_manager.cc:194] Registered new tserver with Master: 57e08ebe49d94c4eb09595ec02c012ab (127.14.61.1:38013)
I20250111 09:03:23.559540 24910 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46904
I20250111 09:03:23.559870 14580 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002237199s
I20250111 09:03:23.565285 24910 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46918:
name: "txn_op_dispatcher_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
I20250111 09:03:23.571825 25021 tablet_service.cc:1467] Processing CreateTablet for tablet 9b2145de504440f79192bafbee8d16ca (DEFAULT_TABLE table=txn_op_dispatcher_test [id=a84da444bc9443e3b678005b981385e6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:23.571825 25020 tablet_service.cc:1467] Processing CreateTablet for tablet c8ecfc9bcf814301b8d08272448608f0 (DEFAULT_TABLE table=txn_op_dispatcher_test [id=a84da444bc9443e3b678005b981385e6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:23.572175 25021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b2145de504440f79192bafbee8d16ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:23.572337 25020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8ecfc9bcf814301b8d08272448608f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:23.574613 25072 tablet_bootstrap.cc:492] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab: Bootstrap starting.
I20250111 09:03:23.575403 25072 tablet_bootstrap.cc:654] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:23.576262 25072 tablet_bootstrap.cc:492] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab: No bootstrap required, opened a new log
I20250111 09:03:23.576335 25072 ts_tablet_manager.cc:1397] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:23.576646 25072 raft_consensus.cc:357] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e08ebe49d94c4eb09595ec02c012ab" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38013 } }
I20250111 09:03:23.576721 25072 raft_consensus.cc:383] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:23.576753 25072 raft_consensus.cc:738] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e08ebe49d94c4eb09595ec02c012ab, State: Initialized, Role: FOLLOWER
I20250111 09:03:23.576859 25072 consensus_queue.cc:260] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e08ebe49d94c4eb09595ec02c012ab" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38013 } }
I20250111 09:03:23.576920 25072 raft_consensus.cc:397] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:23.576959 25072 raft_consensus.cc:491] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:23.577023 25072 raft_consensus.cc:3054] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:23.577674 25072 raft_consensus.cc:513] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e08ebe49d94c4eb09595ec02c012ab" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38013 } }
I20250111 09:03:23.577797 25072 leader_election.cc:304] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57e08ebe49d94c4eb09595ec02c012ab; no voters:
I20250111 09:03:23.577986 25072 leader_election.cc:290] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:23.578073 25074 raft_consensus.cc:2798] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:23.578231 25074 raft_consensus.cc:695] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab [term 1 LEADER]: Becoming Leader. State: Replica: 57e08ebe49d94c4eb09595ec02c012ab, State: Running, Role: LEADER
I20250111 09:03:23.578312 25072 ts_tablet_manager.cc:1428] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:23.578287 25062 heartbeater.cc:502] Master 127.14.61.62:38671 was elected leader, sending a full tablet report...
I20250111 09:03:23.578423 25074 consensus_queue.cc:237] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e08ebe49d94c4eb09595ec02c012ab" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38013 } }
I20250111 09:03:23.578692 25072 tablet_bootstrap.cc:492] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab: Bootstrap starting.
I20250111 09:03:23.579593 24910 catalog_manager.cc:5526] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e08ebe49d94c4eb09595ec02c012ab (127.14.61.1). New cstate: current_term: 1 leader_uuid: "57e08ebe49d94c4eb09595ec02c012ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e08ebe49d94c4eb09595ec02c012ab" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38013 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:23.579876 25072 tablet_bootstrap.cc:654] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:23.580761 25072 tablet_bootstrap.cc:492] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab: No bootstrap required, opened a new log
I20250111 09:03:23.580876 25072 ts_tablet_manager.cc:1397] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:23.581300 25072 raft_consensus.cc:357] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e08ebe49d94c4eb09595ec02c012ab" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38013 } }
I20250111 09:03:23.581382 25072 raft_consensus.cc:383] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:23.581447 25072 raft_consensus.cc:738] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e08ebe49d94c4eb09595ec02c012ab, State: Initialized, Role: FOLLOWER
I20250111 09:03:23.581570 25072 consensus_queue.cc:260] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e08ebe49d94c4eb09595ec02c012ab" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38013 } }
I20250111 09:03:23.581686 25072 raft_consensus.cc:397] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:23.581727 25072 raft_consensus.cc:491] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:23.581771 25072 raft_consensus.cc:3054] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:23.582333 25072 raft_consensus.cc:513] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e08ebe49d94c4eb09595ec02c012ab" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38013 } }
I20250111 09:03:23.582435 25072 leader_election.cc:304] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57e08ebe49d94c4eb09595ec02c012ab; no voters:
I20250111 09:03:23.582516 25072 leader_election.cc:290] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:23.582562 25074 raft_consensus.cc:2798] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:23.582656 25074 raft_consensus.cc:695] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab [term 1 LEADER]: Becoming Leader. State: Replica: 57e08ebe49d94c4eb09595ec02c012ab, State: Running, Role: LEADER
I20250111 09:03:23.582729 25072 ts_tablet_manager.cc:1428] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250111 09:03:23.582751 25074 consensus_queue.cc:237] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e08ebe49d94c4eb09595ec02c012ab" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38013 } }
I20250111 09:03:23.583732 24910 catalog_manager.cc:5526] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e08ebe49d94c4eb09595ec02c012ab (127.14.61.1). New cstate: current_term: 1 leader_uuid: "57e08ebe49d94c4eb09595ec02c012ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e08ebe49d94c4eb09595ec02c012ab" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38013 } health_report { overall_health: HEALTHY } } }
W20250111 09:03:23.587713 25067 txn_manager_proxy_rpc.cc:150] re-attempting BeginTransaction request to one of TxnManagers
I20250111 09:03:24.526540 24910 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46922:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:24.531181 25020 tablet_service.cc:1467] Processing CreateTablet for tablet 34d52467619c4808a89a0ef3832540c4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0dd281b6db9e49189d2d3b5cc55231b4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:03:24.531436 25020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34d52467619c4808a89a0ef3832540c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:24.533154 25090 tablet_bootstrap.cc:492] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab: Bootstrap starting.
I20250111 09:03:24.533938 25090 tablet_bootstrap.cc:654] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:24.534902 25090 tablet_bootstrap.cc:492] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab: No bootstrap required, opened a new log
I20250111 09:03:24.534981 25090 ts_tablet_manager.cc:1397] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:24.535336 25090 raft_consensus.cc:357] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e08ebe49d94c4eb09595ec02c012ab" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38013 } }
I20250111 09:03:24.535415 25090 raft_consensus.cc:383] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:24.535444 25090 raft_consensus.cc:738] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e08ebe49d94c4eb09595ec02c012ab, State: Initialized, Role: FOLLOWER
I20250111 09:03:24.535547 25090 consensus_queue.cc:260] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e08ebe49d94c4eb09595ec02c012ab" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38013 } }
I20250111 09:03:24.535620 25090 raft_consensus.cc:397] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:24.535650 25090 raft_consensus.cc:491] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:24.535686 25090 raft_consensus.cc:3054] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:24.536374 25090 raft_consensus.cc:513] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e08ebe49d94c4eb09595ec02c012ab" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38013 } }
I20250111 09:03:24.536494 25090 leader_election.cc:304] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57e08ebe49d94c4eb09595ec02c012ab; no voters:
I20250111 09:03:24.536669 25090 leader_election.cc:290] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:24.536746 25092 raft_consensus.cc:2798] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:24.536924 25092 raft_consensus.cc:695] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab [term 1 LEADER]: Becoming Leader. State: Replica: 57e08ebe49d94c4eb09595ec02c012ab, State: Running, Role: LEADER
I20250111 09:03:24.537043 25090 ts_tablet_manager.cc:1428] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:24.537091 25092 consensus_queue.cc:237] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e08ebe49d94c4eb09595ec02c012ab" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38013 } }
I20250111 09:03:24.537607 25094 tablet_replica.cc:440] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab: TxnStatusTablet state changed. Reason: New leader 57e08ebe49d94c4eb09595ec02c012ab. Latest consensus state: current_term: 1 leader_uuid: "57e08ebe49d94c4eb09595ec02c012ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e08ebe49d94c4eb09595ec02c012ab" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38013 } } }
I20250111 09:03:24.537679 25094 tablet_replica.cc:443] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:24.537902 25096 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:03:24.537953 25093 tablet_replica.cc:440] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57e08ebe49d94c4eb09595ec02c012ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e08ebe49d94c4eb09595ec02c012ab" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38013 } } }
I20250111 09:03:24.538030 25096 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:03:24.538127 25093 tablet_replica.cc:443] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:24.538479 24910 catalog_manager.cc:5526] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e08ebe49d94c4eb09595ec02c012ab (127.14.61.1). New cstate: current_term: 1 leader_uuid: "57e08ebe49d94c4eb09595ec02c012ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e08ebe49d94c4eb09595ec02c012ab" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 38013 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:26.499521 25104 tablet_replica.cc:331] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab: stopping tablet replica
I20250111 09:03:26.499703 25104 raft_consensus.cc:2238] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:26.499861 25104 raft_consensus.cc:2267] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:26.500412 25104 ts_tablet_manager.cc:1905] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250111 09:03:26.502565 25104 ts_tablet_manager.cc:1918] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20250111 09:03:26.502651 25104 log.cc:1198] T 9b2145de504440f79192bafbee8d16ca P 57e08ebe49d94c4eb09595ec02c012ab: Deleting WAL directory at /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhileReplicaDeleted.1736586070622223-14580-0/minicluster-data/ts-0-root/wals/9b2145de504440f79192bafbee8d16ca
W20250111 09:03:27.509184 25107 tablet_replica.cc:1307] Illegal state: The tablet is not in a running state: STOPPED: cancelling pending write operations
W20250111 09:03:27.509289 25107 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:42194: Illegal state: The tablet is not in a running state: STOPPED
I20250111 09:03:27.509464 25107 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42194 (ReqId={client: c6dd73ba423e4879a7e3e05e2c73ab0f, seq_no=0, attempt_no=0}) took 2006 ms. Trace:
I20250111 09:03:27.509516 25107 rpcz_store.cc:276] 0111 09:03:25.502654 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:25.502705 (+ 51us) service_pool.cc:225] Handling call
0111 09:03:27.509451 (+2006746us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":70,"threads_started":1,"txn-participant-registration.queue_time_us":134}
W20250111 09:03:35.499573 25066 meta_cache.cc:1015] Timed out: LookupRpc { table: 'txn_op_dispatcher_test', partition-key: (HASH (key): 0, RANGE (key): -9223372036854775808), attempt: 1 } failed: LookupRpc timed out after deadline expired: timed out after deadline expired
W20250111 09:03:35.499662 25066 batcher.cc:441] Timed out: Failed to write batch of 1 ops to tablet 9b2145de504440f79192bafbee8d16ca after 123 attempt(s): LookupRpc { table: 'txn_op_dispatcher_test', partition-key: (HASH (key): 0, RANGE (key): -9223372036854775808), attempt: 1 } failed: LookupRpc timed out after deadline expired: timed out after deadline expired
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/integration-tests/txn_write_ops-itest.cc:2052: Failure
Value of: op_status.ToString()
Expected: has substring "STOPPED"
Actual: "Timed out: Failed to write batch of 1 ops to tablet 9b2145de504440f79192bafbee8d16ca after 123 attempt(s): LookupRpc { table: 'txn_op_dispatcher_test', partition-key: (HASH (key): 0, RANGE (key): -9223372036854775808), attempt: 1 } failed: LookupRpc timed out after deadline expired: timed out after deadline expired"
I20250111 09:03:35.500216 14580 tablet_server.cc:178] TabletServer@127.14.61.1:0 shutting down...
I20250111 09:03:35.503314 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250111 09:03:35.503480 14580 tablet_replica.cc:331] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab: stopping tablet replica
I20250111 09:03:35.503585 14580 raft_consensus.cc:2238] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:35.503698 14580 raft_consensus.cc:2267] T c8ecfc9bcf814301b8d08272448608f0 P 57e08ebe49d94c4eb09595ec02c012ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:35.504084 14580 tablet_replica.cc:331] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab: stopping tablet replica
I20250111 09:03:35.504171 14580 raft_consensus.cc:2238] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:35.504273 14580 raft_consensus.cc:2267] T 34d52467619c4808a89a0ef3832540c4 P 57e08ebe49d94c4eb09595ec02c012ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:35.516716 14580 tablet_server.cc:195] TabletServer@127.14.61.1:0 shutdown complete.
I20250111 09:03:35.518703 14580 master.cc:537] Master@127.14.61.62:38671 shutting down...
I20250111 09:03:35.522154 14580 raft_consensus.cc:2238] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:35.522264 14580 raft_consensus.cc:2267] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:35.522323 14580 tablet_replica.cc:331] T 00000000000000000000000000000000 P ef538da7a18b4fadb320f5d722d799bc: stopping tablet replica
I20250111 09:03:35.534000 14580 master.cc:559] Master@127.14.61.62:38671 shutdown complete.
I20250111 09:03:35.536522 14580 test_util.cc:181] -----------------------------------------------
I20250111 09:03:35.536556 14580 test_util.cc:182] Had failures, leaving test files at /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhileReplicaDeleted.1736586070622223-14580-0
[ FAILED ] TxnOpDispatcherITest.TxnWriteWhileReplicaDeleted (12091 ms)
[ RUN ] TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown
I20250111 09:03:35.537292 14580 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.61.62:36387
I20250111 09:03:35.537564 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:35.539836 25111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:35.540529 14580 server_base.cc:1034] running on GCE node
W20250111 09:03:35.541134 25113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:35.541754 25110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:35.541945 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:35.541983 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:35.541997 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586215541997 us; error 0 us; skew 500 ppm
I20250111 09:03:35.542874 14580 webserver.cc:458] Webserver started at http://127.14.61.62:44227/ using document root <none> and password file <none>
I20250111 09:03:35.542991 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:35.543028 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:35.543077 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:35.543433 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown.1736586070622223-14580-0/minicluster-data/master-0-root/instance:
uuid: "84392ea4d9734da0a8a14d6475ed2c0a"
format_stamp: "Formatted at 2025-01-11 09:03:35 on dist-test-slave-nvbt"
I20250111 09:03:35.544726 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:35.545639 25118 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:35.545866 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:35.545933 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown.1736586070622223-14580-0/minicluster-data/master-0-root
uuid: "84392ea4d9734da0a8a14d6475ed2c0a"
format_stamp: "Formatted at 2025-01-11 09:03:35 on dist-test-slave-nvbt"
I20250111 09:03:35.545996 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown.1736586070622223-14580-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown.1736586070622223-14580-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown.1736586070622223-14580-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:35.564622 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:35.564854 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:35.568665 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:36387
I20250111 09:03:35.569535 25180 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:36387 every 8 connection(s)
I20250111 09:03:35.569880 25181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:35.571419 25181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a: Bootstrap starting.
I20250111 09:03:35.572065 25181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:35.572829 25181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a: No bootstrap required, opened a new log
I20250111 09:03:35.573171 25181 raft_consensus.cc:357] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84392ea4d9734da0a8a14d6475ed2c0a" member_type: VOTER }
I20250111 09:03:35.573247 25181 raft_consensus.cc:383] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:35.573279 25181 raft_consensus.cc:738] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84392ea4d9734da0a8a14d6475ed2c0a, State: Initialized, Role: FOLLOWER
I20250111 09:03:35.573395 25181 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84392ea4d9734da0a8a14d6475ed2c0a" member_type: VOTER }
I20250111 09:03:35.573452 25181 raft_consensus.cc:397] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:35.573485 25181 raft_consensus.cc:491] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:35.573529 25181 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:35.574066 25181 raft_consensus.cc:513] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84392ea4d9734da0a8a14d6475ed2c0a" member_type: VOTER }
I20250111 09:03:35.574165 25181 leader_election.cc:304] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84392ea4d9734da0a8a14d6475ed2c0a; no voters:
I20250111 09:03:35.574327 25181 leader_election.cc:290] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:35.574409 25184 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:35.574582 25184 raft_consensus.cc:695] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a [term 1 LEADER]: Becoming Leader. State: Replica: 84392ea4d9734da0a8a14d6475ed2c0a, State: Running, Role: LEADER
I20250111 09:03:35.574672 25181 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:03:35.574699 25184 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84392ea4d9734da0a8a14d6475ed2c0a" member_type: VOTER }
I20250111 09:03:35.575131 25185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84392ea4d9734da0a8a14d6475ed2c0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84392ea4d9734da0a8a14d6475ed2c0a" member_type: VOTER } }
I20250111 09:03:35.575178 25186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84392ea4d9734da0a8a14d6475ed2c0a. Latest consensus state: current_term: 1 leader_uuid: "84392ea4d9734da0a8a14d6475ed2c0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84392ea4d9734da0a8a14d6475ed2c0a" member_type: VOTER } }
I20250111 09:03:35.575309 25185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:35.575325 25186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:35.575732 25192 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:03:35.576447 25192 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:03:35.576617 14580 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250111 09:03:35.578279 25192 catalog_manager.cc:1348] Generated new cluster ID: 476d1d9ceb1e43d18e4c7f9808ce4813
I20250111 09:03:35.578328 25192 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:03:35.587284 25192 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:03:35.587718 25192 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:03:35.595332 25192 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a: Generated new TSK 0
I20250111 09:03:35.595471 25192 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:03:35.602660 25135 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45178:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:35.608745 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:35.610656 25208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:35.611533 25209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:35.612182 25211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:35.612751 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:35.612879 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:35.612916 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:35.612929 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586215612929 us; error 0 us; skew 500 ppm
I20250111 09:03:35.613690 14580 webserver.cc:458] Webserver started at http://127.14.61.1:39761/ using document root <none> and password file <none>
I20250111 09:03:35.613811 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:35.613847 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:35.613911 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:35.614209 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown.1736586070622223-14580-0/minicluster-data/ts-0-root/instance:
uuid: "c40510b56739488988af61e6f6774a06"
format_stamp: "Formatted at 2025-01-11 09:03:35 on dist-test-slave-nvbt"
I20250111 09:03:35.615492 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250111 09:03:35.616278 25216 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:35.616468 14580 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250111 09:03:35.616534 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown.1736586070622223-14580-0/minicluster-data/ts-0-root
uuid: "c40510b56739488988af61e6f6774a06"
format_stamp: "Formatted at 2025-01-11 09:03:35 on dist-test-slave-nvbt"
I20250111 09:03:35.616598 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown.1736586070622223-14580-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown.1736586070622223-14580-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown.1736586070622223-14580-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:35.631098 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:35.631376 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:35.632040 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:03:35.632109 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:35.632160 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:03:35.632182 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:35.637075 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:45703
I20250111 09:03:35.637217 25286 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:45703 every 8 connection(s)
I20250111 09:03:35.637799 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250111 09:03:35.637952 25287 heartbeater.cc:346] Connected to a master server at 127.14.61.62:36387
I20250111 09:03:35.638025 25287 heartbeater.cc:463] Registering TS with master...
I20250111 09:03:35.638188 25287 heartbeater.cc:510] Master 127.14.61.62:36387 requested a full tablet report, sending...
I20250111 09:03:35.638700 25135 ts_manager.cc:194] Registered new tserver with Master: c40510b56739488988af61e6f6774a06 (127.14.61.1:45703)
I20250111 09:03:35.639313 25135 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45184
W20250111 09:03:35.640785 25290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:35.641420 25289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:35.642135 25292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:35.642900 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:35.643036 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:35.643072 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:35.643093 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586215643092 us; error 0 us; skew 500 ppm
I20250111 09:03:35.643807 14580 webserver.cc:458] Webserver started at http://127.14.61.2:41819/ using document root <none> and password file <none>
I20250111 09:03:35.643934 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:35.643980 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:35.644044 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:35.644341 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown.1736586070622223-14580-0/minicluster-data/ts-1-root/instance:
uuid: "9c26b2b7505541018c43df669b6a8479"
format_stamp: "Formatted at 2025-01-11 09:03:35 on dist-test-slave-nvbt"
I20250111 09:03:35.645767 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250111 09:03:35.646708 25297 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:35.646929 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:35.646994 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown.1736586070622223-14580-0/minicluster-data/ts-1-root
uuid: "9c26b2b7505541018c43df669b6a8479"
format_stamp: "Formatted at 2025-01-11 09:03:35 on dist-test-slave-nvbt"
I20250111 09:03:35.647059 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown.1736586070622223-14580-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown.1736586070622223-14580-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown.1736586070622223-14580-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:35.661326 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:35.661651 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:35.662351 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:03:35.662438 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:35.662504 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:03:35.662554 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:35.668541 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.2:45241
I20250111 09:03:35.668576 25362 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.2:45241 every 8 connection(s)
I20250111 09:03:35.669423 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250111 09:03:35.669638 25363 heartbeater.cc:346] Connected to a master server at 127.14.61.62:36387
I20250111 09:03:35.669780 25363 heartbeater.cc:463] Registering TS with master...
I20250111 09:03:35.670037 25363 heartbeater.cc:510] Master 127.14.61.62:36387 requested a full tablet report, sending...
I20250111 09:03:35.670671 25135 ts_manager.cc:194] Registered new tserver with Master: 9c26b2b7505541018c43df669b6a8479 (127.14.61.2:45241)
I20250111 09:03:35.671397 25135 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45196
I20250111 09:03:35.672170 14580 server_base.cc:1034] running on GCE node
W20250111 09:03:35.672899 25370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:35.673645 25373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:35.674312 25371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:35.674537 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:35.674597 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:35.674620 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586215674620 us; error 0 us; skew 500 ppm
I20250111 09:03:35.675640 14580 webserver.cc:458] Webserver started at http://127.14.61.3:40729/ using document root <none> and password file <none>
I20250111 09:03:35.675839 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:35.675891 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:35.675966 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:35.676383 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown.1736586070622223-14580-0/minicluster-data/ts-2-root/instance:
uuid: "8cac247555284a81a92a14ca81b24757"
format_stamp: "Formatted at 2025-01-11 09:03:35 on dist-test-slave-nvbt"
I20250111 09:03:35.677727 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250111 09:03:35.678436 25378 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:35.678638 14580 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250111 09:03:35.678699 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown.1736586070622223-14580-0/minicluster-data/ts-2-root
uuid: "8cac247555284a81a92a14ca81b24757"
format_stamp: "Formatted at 2025-01-11 09:03:35 on dist-test-slave-nvbt"
I20250111 09:03:35.678761 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown.1736586070622223-14580-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown.1736586070622223-14580-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown.1736586070622223-14580-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:35.683621 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:35.683916 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:35.684489 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:03:35.684530 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:35.684572 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:03:35.684607 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:35.690927 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.3:37641
I20250111 09:03:35.690963 25448 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.3:37641 every 8 connection(s)
I20250111 09:03:35.691830 25449 heartbeater.cc:346] Connected to a master server at 127.14.61.62:36387
I20250111 09:03:35.691917 25449 heartbeater.cc:463] Registering TS with master...
I20250111 09:03:35.692082 25449 heartbeater.cc:510] Master 127.14.61.62:36387 requested a full tablet report, sending...
I20250111 09:03:35.692588 25135 ts_manager.cc:194] Registered new tserver with Master: 8cac247555284a81a92a14ca81b24757 (127.14.61.3:37641)
I20250111 09:03:35.692966 25135 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45202
I20250111 09:03:35.693614 14580 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.002272507s
I20250111 09:03:35.699103 25135 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45210:
name: "txn_op_dispatcher_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
W20250111 09:03:35.699486 25135 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table txn_op_dispatcher_test in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250111 09:03:35.707670 25246 tablet_service.cc:1467] Processing CreateTablet for tablet e7a5d326eda64000b26dde512408aabb (DEFAULT_TABLE table=txn_op_dispatcher_test [id=48da163d63b74066b9d59dc07e7c2ab0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:35.707727 25245 tablet_service.cc:1467] Processing CreateTablet for tablet 0dc393153ba8410b80d2ecf283ff996e (DEFAULT_TABLE table=txn_op_dispatcher_test [id=48da163d63b74066b9d59dc07e7c2ab0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:35.707960 25246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7a5d326eda64000b26dde512408aabb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:35.708089 25245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dc393153ba8410b80d2ecf283ff996e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:35.708137 25327 tablet_service.cc:1467] Processing CreateTablet for tablet e7a5d326eda64000b26dde512408aabb (DEFAULT_TABLE table=txn_op_dispatcher_test [id=48da163d63b74066b9d59dc07e7c2ab0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:35.708333 25408 tablet_service.cc:1467] Processing CreateTablet for tablet e7a5d326eda64000b26dde512408aabb (DEFAULT_TABLE table=txn_op_dispatcher_test [id=48da163d63b74066b9d59dc07e7c2ab0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:35.708566 25408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7a5d326eda64000b26dde512408aabb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:35.708547 25326 tablet_service.cc:1467] Processing CreateTablet for tablet 0dc393153ba8410b80d2ecf283ff996e (DEFAULT_TABLE table=txn_op_dispatcher_test [id=48da163d63b74066b9d59dc07e7c2ab0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:35.710364 25464 tablet_bootstrap.cc:492] T e7a5d326eda64000b26dde512408aabb P 8cac247555284a81a92a14ca81b24757: Bootstrap starting.
I20250111 09:03:35.710505 25407 tablet_service.cc:1467] Processing CreateTablet for tablet 0dc393153ba8410b80d2ecf283ff996e (DEFAULT_TABLE table=txn_op_dispatcher_test [id=48da163d63b74066b9d59dc07e7c2ab0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:35.710716 25407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dc393153ba8410b80d2ecf283ff996e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:35.711376 25464 tablet_bootstrap.cc:654] T e7a5d326eda64000b26dde512408aabb P 8cac247555284a81a92a14ca81b24757: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:35.711971 25463 tablet_bootstrap.cc:492] T 0dc393153ba8410b80d2ecf283ff996e P c40510b56739488988af61e6f6774a06: Bootstrap starting.
I20250111 09:03:35.708343 25327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7a5d326eda64000b26dde512408aabb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:35.712320 25326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dc393153ba8410b80d2ecf283ff996e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:35.712823 25463 tablet_bootstrap.cc:654] T 0dc393153ba8410b80d2ecf283ff996e P c40510b56739488988af61e6f6774a06: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:35.713277 25464 tablet_bootstrap.cc:492] T e7a5d326eda64000b26dde512408aabb P 8cac247555284a81a92a14ca81b24757: No bootstrap required, opened a new log
I20250111 09:03:35.713426 25464 ts_tablet_manager.cc:1397] T e7a5d326eda64000b26dde512408aabb P 8cac247555284a81a92a14ca81b24757: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250111 09:03:35.714130 25464 raft_consensus.cc:357] T e7a5d326eda64000b26dde512408aabb P 8cac247555284a81a92a14ca81b24757 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } } peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } }
I20250111 09:03:35.714236 25464 raft_consensus.cc:383] T e7a5d326eda64000b26dde512408aabb P 8cac247555284a81a92a14ca81b24757 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:35.714282 25464 raft_consensus.cc:738] T e7a5d326eda64000b26dde512408aabb P 8cac247555284a81a92a14ca81b24757 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cac247555284a81a92a14ca81b24757, State: Initialized, Role: FOLLOWER
I20250111 09:03:35.714408 25464 consensus_queue.cc:260] T e7a5d326eda64000b26dde512408aabb P 8cac247555284a81a92a14ca81b24757 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } } peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } }
I20250111 09:03:35.714711 25464 ts_tablet_manager.cc:1428] T e7a5d326eda64000b26dde512408aabb P 8cac247555284a81a92a14ca81b24757: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:35.714738 25449 heartbeater.cc:502] Master 127.14.61.62:36387 was elected leader, sending a full tablet report...
I20250111 09:03:35.714823 25464 tablet_bootstrap.cc:492] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757: Bootstrap starting.
I20250111 09:03:35.715638 25464 tablet_bootstrap.cc:654] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:35.716499 25467 tablet_bootstrap.cc:492] T e7a5d326eda64000b26dde512408aabb P 9c26b2b7505541018c43df669b6a8479: Bootstrap starting.
I20250111 09:03:35.716686 25463 tablet_bootstrap.cc:492] T 0dc393153ba8410b80d2ecf283ff996e P c40510b56739488988af61e6f6774a06: No bootstrap required, opened a new log
I20250111 09:03:35.716742 25464 tablet_bootstrap.cc:492] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757: No bootstrap required, opened a new log
I20250111 09:03:35.716751 25463 ts_tablet_manager.cc:1397] T 0dc393153ba8410b80d2ecf283ff996e P c40510b56739488988af61e6f6774a06: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20250111 09:03:35.716810 25464 ts_tablet_manager.cc:1397] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:35.717164 25464 raft_consensus.cc:357] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } }
I20250111 09:03:35.717185 25463 raft_consensus.cc:357] T 0dc393153ba8410b80d2ecf283ff996e P c40510b56739488988af61e6f6774a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } }
I20250111 09:03:35.717262 25464 raft_consensus.cc:383] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:35.717269 25463 raft_consensus.cc:383] T 0dc393153ba8410b80d2ecf283ff996e P c40510b56739488988af61e6f6774a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:35.717307 25464 raft_consensus.cc:738] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cac247555284a81a92a14ca81b24757, State: Initialized, Role: FOLLOWER
I20250111 09:03:35.717329 25463 raft_consensus.cc:738] T 0dc393153ba8410b80d2ecf283ff996e P c40510b56739488988af61e6f6774a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40510b56739488988af61e6f6774a06, State: Initialized, Role: FOLLOWER
I20250111 09:03:35.717473 25467 tablet_bootstrap.cc:654] T e7a5d326eda64000b26dde512408aabb P 9c26b2b7505541018c43df669b6a8479: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:35.717430 25464 consensus_queue.cc:260] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } }
I20250111 09:03:35.717442 25463 consensus_queue.cc:260] T 0dc393153ba8410b80d2ecf283ff996e P c40510b56739488988af61e6f6774a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } }
I20250111 09:03:35.717695 25464 ts_tablet_manager.cc:1428] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:35.717787 25463 ts_tablet_manager.cc:1428] T 0dc393153ba8410b80d2ecf283ff996e P c40510b56739488988af61e6f6774a06: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:35.717922 25463 tablet_bootstrap.cc:492] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06: Bootstrap starting.
I20250111 09:03:35.717993 25287 heartbeater.cc:502] Master 127.14.61.62:36387 was elected leader, sending a full tablet report...
I20250111 09:03:35.718797 25463 tablet_bootstrap.cc:654] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:35.719231 25467 tablet_bootstrap.cc:492] T e7a5d326eda64000b26dde512408aabb P 9c26b2b7505541018c43df669b6a8479: No bootstrap required, opened a new log
I20250111 09:03:35.719308 25467 ts_tablet_manager.cc:1397] T e7a5d326eda64000b26dde512408aabb P 9c26b2b7505541018c43df669b6a8479: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250111 09:03:35.719677 25467 raft_consensus.cc:357] T e7a5d326eda64000b26dde512408aabb P 9c26b2b7505541018c43df669b6a8479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } } peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } }
I20250111 09:03:35.719779 25467 raft_consensus.cc:383] T e7a5d326eda64000b26dde512408aabb P 9c26b2b7505541018c43df669b6a8479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:35.719820 25467 raft_consensus.cc:738] T e7a5d326eda64000b26dde512408aabb P 9c26b2b7505541018c43df669b6a8479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c26b2b7505541018c43df669b6a8479, State: Initialized, Role: FOLLOWER
I20250111 09:03:35.719867 25463 tablet_bootstrap.cc:492] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06: No bootstrap required, opened a new log
I20250111 09:03:35.719939 25463 ts_tablet_manager.cc:1397] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:35.719938 25467 consensus_queue.cc:260] T e7a5d326eda64000b26dde512408aabb P 9c26b2b7505541018c43df669b6a8479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } } peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } }
I20250111 09:03:35.720319 25467 ts_tablet_manager.cc:1428] T e7a5d326eda64000b26dde512408aabb P 9c26b2b7505541018c43df669b6a8479: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:35.720342 25363 heartbeater.cc:502] Master 127.14.61.62:36387 was elected leader, sending a full tablet report...
I20250111 09:03:35.720348 25463 raft_consensus.cc:357] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } } peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } }
I20250111 09:03:35.720443 25467 tablet_bootstrap.cc:492] T 0dc393153ba8410b80d2ecf283ff996e P 9c26b2b7505541018c43df669b6a8479: Bootstrap starting.
I20250111 09:03:35.720484 25463 raft_consensus.cc:383] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:35.720516 25463 raft_consensus.cc:738] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40510b56739488988af61e6f6774a06, State: Initialized, Role: FOLLOWER
I20250111 09:03:35.720614 25463 consensus_queue.cc:260] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } } peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } }
I20250111 09:03:35.720837 25463 ts_tablet_manager.cc:1428] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250111 09:03:35.721307 25467 tablet_bootstrap.cc:654] T 0dc393153ba8410b80d2ecf283ff996e P 9c26b2b7505541018c43df669b6a8479: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:35.722137 25467 tablet_bootstrap.cc:492] T 0dc393153ba8410b80d2ecf283ff996e P 9c26b2b7505541018c43df669b6a8479: No bootstrap required, opened a new log
I20250111 09:03:35.722218 25467 ts_tablet_manager.cc:1397] T 0dc393153ba8410b80d2ecf283ff996e P 9c26b2b7505541018c43df669b6a8479: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:35.722632 25467 raft_consensus.cc:357] T 0dc393153ba8410b80d2ecf283ff996e P 9c26b2b7505541018c43df669b6a8479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } }
I20250111 09:03:35.722723 25467 raft_consensus.cc:383] T 0dc393153ba8410b80d2ecf283ff996e P 9c26b2b7505541018c43df669b6a8479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:35.722774 25467 raft_consensus.cc:738] T 0dc393153ba8410b80d2ecf283ff996e P 9c26b2b7505541018c43df669b6a8479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c26b2b7505541018c43df669b6a8479, State: Initialized, Role: FOLLOWER
I20250111 09:03:35.722882 25467 consensus_queue.cc:260] T 0dc393153ba8410b80d2ecf283ff996e P 9c26b2b7505541018c43df669b6a8479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } }
I20250111 09:03:35.723111 25467 ts_tablet_manager.cc:1428] T 0dc393153ba8410b80d2ecf283ff996e P 9c26b2b7505541018c43df669b6a8479: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250111 09:03:35.849928 25468 raft_consensus.cc:491] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:03:35.850044 25468 raft_consensus.cc:513] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } }
I20250111 09:03:35.850879 25468 leader_election.cc:290] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40510b56739488988af61e6f6774a06 (127.14.61.1:45703), 9c26b2b7505541018c43df669b6a8479 (127.14.61.2:45241)
I20250111 09:03:35.853996 25337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dc393153ba8410b80d2ecf283ff996e" candidate_uuid: "8cac247555284a81a92a14ca81b24757" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c26b2b7505541018c43df669b6a8479" is_pre_election: true
I20250111 09:03:35.853998 25256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dc393153ba8410b80d2ecf283ff996e" candidate_uuid: "8cac247555284a81a92a14ca81b24757" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40510b56739488988af61e6f6774a06" is_pre_election: true
I20250111 09:03:35.854182 25337 raft_consensus.cc:2463] T 0dc393153ba8410b80d2ecf283ff996e P 9c26b2b7505541018c43df669b6a8479 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cac247555284a81a92a14ca81b24757 in term 0.
I20250111 09:03:35.854182 25256 raft_consensus.cc:2463] T 0dc393153ba8410b80d2ecf283ff996e P c40510b56739488988af61e6f6774a06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cac247555284a81a92a14ca81b24757 in term 0.
I20250111 09:03:35.854588 25382 leader_election.cc:304] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cac247555284a81a92a14ca81b24757, c40510b56739488988af61e6f6774a06; no voters:
I20250111 09:03:35.854735 25468 raft_consensus.cc:2798] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:03:35.854789 25468 raft_consensus.cc:491] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:03:35.854815 25468 raft_consensus.cc:3054] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:35.855636 25468 raft_consensus.cc:513] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } }
I20250111 09:03:35.855957 25468 leader_election.cc:290] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 [CANDIDATE]: Term 1 election: Requested vote from peers c40510b56739488988af61e6f6774a06 (127.14.61.1:45703), 9c26b2b7505541018c43df669b6a8479 (127.14.61.2:45241)
I20250111 09:03:35.856240 25256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dc393153ba8410b80d2ecf283ff996e" candidate_uuid: "8cac247555284a81a92a14ca81b24757" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40510b56739488988af61e6f6774a06"
I20250111 09:03:35.856245 25337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dc393153ba8410b80d2ecf283ff996e" candidate_uuid: "8cac247555284a81a92a14ca81b24757" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c26b2b7505541018c43df669b6a8479"
I20250111 09:03:35.856333 25256 raft_consensus.cc:3054] T 0dc393153ba8410b80d2ecf283ff996e P c40510b56739488988af61e6f6774a06 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:35.856336 25337 raft_consensus.cc:3054] T 0dc393153ba8410b80d2ecf283ff996e P 9c26b2b7505541018c43df669b6a8479 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:35.857077 25337 raft_consensus.cc:2463] T 0dc393153ba8410b80d2ecf283ff996e P 9c26b2b7505541018c43df669b6a8479 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cac247555284a81a92a14ca81b24757 in term 1.
I20250111 09:03:35.857093 25256 raft_consensus.cc:2463] T 0dc393153ba8410b80d2ecf283ff996e P c40510b56739488988af61e6f6774a06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cac247555284a81a92a14ca81b24757 in term 1.
I20250111 09:03:35.857328 25379 leader_election.cc:304] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cac247555284a81a92a14ca81b24757, 9c26b2b7505541018c43df669b6a8479; no voters:
I20250111 09:03:35.857471 25468 raft_consensus.cc:2798] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:35.857637 25468 raft_consensus.cc:695] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 [term 1 LEADER]: Becoming Leader. State: Replica: 8cac247555284a81a92a14ca81b24757, State: Running, Role: LEADER
I20250111 09:03:35.857762 25468 consensus_queue.cc:237] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } }
I20250111 09:03:35.859267 25135 catalog_manager.cc:5526] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8cac247555284a81a92a14ca81b24757 (127.14.61.3). New cstate: current_term: 1 leader_uuid: "8cac247555284a81a92a14ca81b24757" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:35.860291 25470 raft_consensus.cc:491] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:03:35.860375 25470 raft_consensus.cc:513] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } } peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } }
I20250111 09:03:35.860755 25470 leader_election.cc:290] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c26b2b7505541018c43df669b6a8479 (127.14.61.2:45241), 8cac247555284a81a92a14ca81b24757 (127.14.61.3:37641)
I20250111 09:03:35.863466 25337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7a5d326eda64000b26dde512408aabb" candidate_uuid: "c40510b56739488988af61e6f6774a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c26b2b7505541018c43df669b6a8479" is_pre_election: true
I20250111 09:03:35.863556 25337 raft_consensus.cc:2463] T e7a5d326eda64000b26dde512408aabb P 9c26b2b7505541018c43df669b6a8479 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40510b56739488988af61e6f6774a06 in term 0.
I20250111 09:03:35.863801 25217 leader_election.cc:304] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c26b2b7505541018c43df669b6a8479, c40510b56739488988af61e6f6774a06; no voters:
I20250111 09:03:35.863852 25415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7a5d326eda64000b26dde512408aabb" candidate_uuid: "c40510b56739488988af61e6f6774a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cac247555284a81a92a14ca81b24757" is_pre_election: true
I20250111 09:03:35.863936 25470 raft_consensus.cc:2798] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:03:35.863983 25470 raft_consensus.cc:491] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:03:35.863962 25415 raft_consensus.cc:2463] T e7a5d326eda64000b26dde512408aabb P 8cac247555284a81a92a14ca81b24757 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40510b56739488988af61e6f6774a06 in term 0.
I20250111 09:03:35.864028 25470 raft_consensus.cc:3054] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:35.864640 25470 raft_consensus.cc:513] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } } peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } }
I20250111 09:03:35.864876 25470 leader_election.cc:290] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 [CANDIDATE]: Term 1 election: Requested vote from peers 9c26b2b7505541018c43df669b6a8479 (127.14.61.2:45241), 8cac247555284a81a92a14ca81b24757 (127.14.61.3:37641)
I20250111 09:03:35.865195 25415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7a5d326eda64000b26dde512408aabb" candidate_uuid: "c40510b56739488988af61e6f6774a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cac247555284a81a92a14ca81b24757"
I20250111 09:03:35.865203 25337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7a5d326eda64000b26dde512408aabb" candidate_uuid: "c40510b56739488988af61e6f6774a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c26b2b7505541018c43df669b6a8479"
I20250111 09:03:35.865291 25415 raft_consensus.cc:3054] T e7a5d326eda64000b26dde512408aabb P 8cac247555284a81a92a14ca81b24757 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:35.865331 25337 raft_consensus.cc:3054] T e7a5d326eda64000b26dde512408aabb P 9c26b2b7505541018c43df669b6a8479 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:35.865952 25337 raft_consensus.cc:2463] T e7a5d326eda64000b26dde512408aabb P 9c26b2b7505541018c43df669b6a8479 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40510b56739488988af61e6f6774a06 in term 1.
I20250111 09:03:35.865998 25415 raft_consensus.cc:2463] T e7a5d326eda64000b26dde512408aabb P 8cac247555284a81a92a14ca81b24757 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40510b56739488988af61e6f6774a06 in term 1.
I20250111 09:03:35.866215 25217 leader_election.cc:304] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c26b2b7505541018c43df669b6a8479, c40510b56739488988af61e6f6774a06; no voters:
I20250111 09:03:35.866389 25470 raft_consensus.cc:2798] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:35.866573 25470 raft_consensus.cc:695] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 [term 1 LEADER]: Becoming Leader. State: Replica: c40510b56739488988af61e6f6774a06, State: Running, Role: LEADER
I20250111 09:03:35.866698 25470 consensus_queue.cc:237] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } } peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } }
I20250111 09:03:35.867880 25135 catalog_manager.cc:5526] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 reported cstate change: term changed from 0 to 1, leader changed from <none> to c40510b56739488988af61e6f6774a06 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "c40510b56739488988af61e6f6774a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } health_report { overall_health: HEALTHY } } }
W20250111 09:03:35.887727 25288 tablet.cc:2367] T 0dc393153ba8410b80d2ecf283ff996e P c40510b56739488988af61e6f6774a06: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250111 09:03:35.890291 25455 txn_manager_proxy_rpc.cc:150] re-attempting BeginTransaction request to one of TxnManagers
I20250111 09:03:36.297847 25468 consensus_queue.cc:1035] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:03:36.301146 25484 consensus_queue.cc:1035] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:03:36.338984 25470 consensus_queue.cc:1035] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:03:36.341293 25479 consensus_queue.cc:1035] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:03:36.614503 25135 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45224:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20250111 09:03:36.614997 25135 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250111 09:03:36.619167 25245 tablet_service.cc:1467] Processing CreateTablet for tablet 778338e896ef4007aaf16fdc91644157 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6acab50f8b774c11be38a66aae931486]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:03:36.619362 25407 tablet_service.cc:1467] Processing CreateTablet for tablet 778338e896ef4007aaf16fdc91644157 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6acab50f8b774c11be38a66aae931486]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:03:36.619462 25245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 778338e896ef4007aaf16fdc91644157. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:36.619611 25407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 778338e896ef4007aaf16fdc91644157. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:36.621086 25499 tablet_bootstrap.cc:492] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06: Bootstrap starting.
I20250111 09:03:36.621219 25326 tablet_service.cc:1467] Processing CreateTablet for tablet 778338e896ef4007aaf16fdc91644157 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6acab50f8b774c11be38a66aae931486]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:03:36.621241 25500 tablet_bootstrap.cc:492] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757: Bootstrap starting.
I20250111 09:03:36.621450 25326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 778338e896ef4007aaf16fdc91644157. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:36.622048 25500 tablet_bootstrap.cc:654] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:36.622105 25499 tablet_bootstrap.cc:654] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:36.622812 25500 tablet_bootstrap.cc:492] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757: No bootstrap required, opened a new log
I20250111 09:03:36.622865 25500 ts_tablet_manager.cc:1397] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:36.623178 25500 raft_consensus.cc:357] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } }
I20250111 09:03:36.623359 25500 raft_consensus.cc:383] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:36.623190 25503 tablet_bootstrap.cc:492] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479: Bootstrap starting.
I20250111 09:03:36.623462 25500 raft_consensus.cc:738] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cac247555284a81a92a14ca81b24757, State: Initialized, Role: FOLLOWER
I20250111 09:03:36.623838 25499 tablet_bootstrap.cc:492] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06: No bootstrap required, opened a new log
I20250111 09:03:36.624043 25499 ts_tablet_manager.cc:1397] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250111 09:03:36.624009 25500 consensus_queue.cc:260] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } }
I20250111 09:03:36.624235 25476 tablet_replica.cc:440] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } } }
I20250111 09:03:36.624300 25476 tablet_replica.cc:443] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:03:36.624483 25500 ts_tablet_manager.cc:1428] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:36.624696 25503 tablet_bootstrap.cc:654] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:36.624779 25499 raft_consensus.cc:357] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } }
I20250111 09:03:36.624878 25499 raft_consensus.cc:383] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:36.624933 25499 raft_consensus.cc:738] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40510b56739488988af61e6f6774a06, State: Initialized, Role: FOLLOWER
I20250111 09:03:36.625115 25499 consensus_queue.cc:260] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } }
I20250111 09:03:36.625349 25479 tablet_replica.cc:440] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } } }
I20250111 09:03:36.625430 25499 ts_tablet_manager.cc:1428] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:36.625429 25479 tablet_replica.cc:443] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:03:36.626446 25503 tablet_bootstrap.cc:492] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479: No bootstrap required, opened a new log
I20250111 09:03:36.626519 25503 ts_tablet_manager.cc:1397] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20250111 09:03:36.626951 25503 raft_consensus.cc:357] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } }
I20250111 09:03:36.627045 25503 raft_consensus.cc:383] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:36.627085 25503 raft_consensus.cc:738] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c26b2b7505541018c43df669b6a8479, State: Initialized, Role: FOLLOWER
I20250111 09:03:36.627214 25503 consensus_queue.cc:260] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } }
I20250111 09:03:36.627418 25481 tablet_replica.cc:440] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } } }
I20250111 09:03:36.627470 25503 ts_tablet_manager.cc:1428] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250111 09:03:36.627501 25481 tablet_replica.cc:443] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:03:36.698417 25481 raft_consensus.cc:491] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 09:03:36.698518 25481 raft_consensus.cc:513] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } }
I20250111 09:03:36.698895 25481 leader_election.cc:290] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40510b56739488988af61e6f6774a06 (127.14.61.1:45703), 8cac247555284a81a92a14ca81b24757 (127.14.61.3:37641)
I20250111 09:03:36.702018 25415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "778338e896ef4007aaf16fdc91644157" candidate_uuid: "9c26b2b7505541018c43df669b6a8479" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cac247555284a81a92a14ca81b24757" is_pre_election: true
I20250111 09:03:36.702023 25256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "778338e896ef4007aaf16fdc91644157" candidate_uuid: "9c26b2b7505541018c43df669b6a8479" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40510b56739488988af61e6f6774a06" is_pre_election: true
I20250111 09:03:36.702142 25415 raft_consensus.cc:2463] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c26b2b7505541018c43df669b6a8479 in term 0.
I20250111 09:03:36.702169 25256 raft_consensus.cc:2463] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c26b2b7505541018c43df669b6a8479 in term 0.
I20250111 09:03:36.702378 25298 leader_election.cc:304] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cac247555284a81a92a14ca81b24757, 9c26b2b7505541018c43df669b6a8479; no voters:
I20250111 09:03:36.702513 25481 raft_consensus.cc:2798] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 09:03:36.702570 25481 raft_consensus.cc:491] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 09:03:36.702607 25481 raft_consensus.cc:3054] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:36.703349 25481 raft_consensus.cc:513] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } }
I20250111 09:03:36.703614 25481 leader_election.cc:290] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 [CANDIDATE]: Term 1 election: Requested vote from peers c40510b56739488988af61e6f6774a06 (127.14.61.1:45703), 8cac247555284a81a92a14ca81b24757 (127.14.61.3:37641)
I20250111 09:03:36.703867 25256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "778338e896ef4007aaf16fdc91644157" candidate_uuid: "9c26b2b7505541018c43df669b6a8479" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40510b56739488988af61e6f6774a06"
I20250111 09:03:36.703919 25415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "778338e896ef4007aaf16fdc91644157" candidate_uuid: "9c26b2b7505541018c43df669b6a8479" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cac247555284a81a92a14ca81b24757"
I20250111 09:03:36.703961 25256 raft_consensus.cc:3054] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:36.704056 25415 raft_consensus.cc:3054] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:36.704732 25256 raft_consensus.cc:2463] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c26b2b7505541018c43df669b6a8479 in term 1.
I20250111 09:03:36.704747 25415 raft_consensus.cc:2463] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c26b2b7505541018c43df669b6a8479 in term 1.
I20250111 09:03:36.704958 25298 leader_election.cc:304] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cac247555284a81a92a14ca81b24757, 9c26b2b7505541018c43df669b6a8479; no voters:
I20250111 09:03:36.705157 25481 raft_consensus.cc:2798] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:36.705339 25481 raft_consensus.cc:695] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 [term 1 LEADER]: Becoming Leader. State: Replica: 9c26b2b7505541018c43df669b6a8479, State: Running, Role: LEADER
I20250111 09:03:36.705477 25481 consensus_queue.cc:237] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } }
I20250111 09:03:36.706072 25509 tablet_replica.cc:440] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479: TxnStatusTablet state changed. Reason: New leader 9c26b2b7505541018c43df669b6a8479. Latest consensus state: current_term: 1 leader_uuid: "9c26b2b7505541018c43df669b6a8479" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } } }
I20250111 09:03:36.706190 25509 tablet_replica.cc:443] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:36.706422 25511 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:03:36.706540 25511 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:03:36.706900 25133 catalog_manager.cc:5526] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c26b2b7505541018c43df669b6a8479 (127.14.61.2). New cstate: current_term: 1 leader_uuid: "9c26b2b7505541018c43df669b6a8479" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } health_report { overall_health: UNKNOWN } } }
I20250111 09:03:36.739506 25415 raft_consensus.cc:1270] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [term 1 FOLLOWER]: Refusing update from remote peer 9c26b2b7505541018c43df669b6a8479: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:03:36.739506 25256 raft_consensus.cc:1270] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06 [term 1 FOLLOWER]: Refusing update from remote peer 9c26b2b7505541018c43df669b6a8479: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250111 09:03:36.739883 25509 consensus_queue.cc:1035] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:03:36.740010 25481 consensus_queue.cc:1035] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 09:03:36.741024 25468 tablet_replica.cc:440] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757: TxnStatusTablet state changed. Reason: New leader 9c26b2b7505541018c43df669b6a8479. Latest consensus state: current_term: 1 leader_uuid: "9c26b2b7505541018c43df669b6a8479" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } } }
I20250111 09:03:36.741112 25468 tablet_replica.cc:443] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:03:36.741755 25479 tablet_replica.cc:440] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06: TxnStatusTablet state changed. Reason: New leader 9c26b2b7505541018c43df669b6a8479. Latest consensus state: current_term: 1 leader_uuid: "9c26b2b7505541018c43df669b6a8479" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } } }
I20250111 09:03:36.741843 25479 tablet_replica.cc:443] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:03:36.741816 25485 mvcc.cc:204] Tried to move back new op lower bound from 7113057143762317312 to 7113057143626964992. Current Snapshot: MvccSnapshot[applied={T|T < 7113057143762317312}]
I20250111 09:03:36.741973 25509 tablet_replica.cc:440] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c26b2b7505541018c43df669b6a8479" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } } }
I20250111 09:03:36.742045 25509 tablet_replica.cc:443] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:36.742522 25468 tablet_replica.cc:440] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c26b2b7505541018c43df669b6a8479" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } } }
I20250111 09:03:36.742595 25468 tablet_replica.cc:443] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:03:36.744453 25479 tablet_replica.cc:440] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c26b2b7505541018c43df669b6a8479" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } } }
I20250111 09:03:36.744557 25479 tablet_replica.cc:443] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06: This TxnStatusTablet replica's current role is: FOLLOWER
I20250111 09:03:36.744660 25481 tablet_replica.cc:440] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c26b2b7505541018c43df669b6a8479" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } } }
I20250111 09:03:36.744727 25481 tablet_replica.cc:443] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:37.744379 25519 tablet_replica.cc:331] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06: stopping tablet replica
I20250111 09:03:37.744576 25519 raft_consensus.cc:2238] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:37.744599 25524 tablet_replica.cc:331] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757: stopping tablet replica
I20250111 09:03:37.744616 25523 tablet_replica.cc:331] T e7a5d326eda64000b26dde512408aabb P 9c26b2b7505541018c43df669b6a8479: stopping tablet replica
I20250111 09:03:37.744721 25524 raft_consensus.cc:2238] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:37.744617 25525 tablet_replica.cc:331] T e7a5d326eda64000b26dde512408aabb P 8cac247555284a81a92a14ca81b24757: stopping tablet replica
I20250111 09:03:37.744868 25522 tablet_replica.cc:331] T 0dc393153ba8410b80d2ecf283ff996e P 9c26b2b7505541018c43df669b6a8479: stopping tablet replica
I20250111 09:03:37.744923 25521 tablet_replica.cc:331] T 0dc393153ba8410b80d2ecf283ff996e P c40510b56739488988af61e6f6774a06: stopping tablet replica
I20250111 09:03:37.744968 25519 raft_consensus.cc:2267] T e7a5d326eda64000b26dde512408aabb P c40510b56739488988af61e6f6774a06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:37.744997 25521 raft_consensus.cc:2238] T 0dc393153ba8410b80d2ecf283ff996e P c40510b56739488988af61e6f6774a06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250111 09:03:37.745090 25523 raft_consensus.cc:2238] T e7a5d326eda64000b26dde512408aabb P 9c26b2b7505541018c43df669b6a8479 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250111 09:03:37.745059 25522 raft_consensus.cc:2238] T 0dc393153ba8410b80d2ecf283ff996e P 9c26b2b7505541018c43df669b6a8479 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250111 09:03:37.745185 25521 raft_consensus.cc:2267] T 0dc393153ba8410b80d2ecf283ff996e P c40510b56739488988af61e6f6774a06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:37.745208 25523 raft_consensus.cc:2267] T e7a5d326eda64000b26dde512408aabb P 9c26b2b7505541018c43df669b6a8479 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:37.745057 25524 raft_consensus.cc:2267] T 0dc393153ba8410b80d2ecf283ff996e P 8cac247555284a81a92a14ca81b24757 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:37.745218 25522 raft_consensus.cc:2267] T 0dc393153ba8410b80d2ecf283ff996e P 9c26b2b7505541018c43df669b6a8479 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:37.744980 25525 raft_consensus.cc:2238] T e7a5d326eda64000b26dde512408aabb P 8cac247555284a81a92a14ca81b24757 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250111 09:03:37.745818 25525 raft_consensus.cc:2267] T e7a5d326eda64000b26dde512408aabb P 8cac247555284a81a92a14ca81b24757 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250111 09:03:38.756371 25528 tablet_replica.cc:1307] Illegal state: The tablet is not in a running state: SHUTDOWN: cancelling pending write operations
W20250111 09:03:38.756565 25528 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:44392: Illegal state: The tablet is not in a running state: SHUTDOWN
I20250111 09:03:38.756714 25528 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:44392 (ReqId={client: 0eb9599b32204b8fa76d317cfa3976b9, seq_no=0, attempt_no=0}) took 2008 ms. Trace:
I20250111 09:03:38.756747 25528 rpcz_store.cc:276] 0111 09:03:36.748586 (+ 0us) service_pool.cc:168] Inserting onto call queue
0111 09:03:36.748631 (+ 45us) service_pool.cc:225] Handling call
0111 09:03:38.756706 (+2008075us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":75,"threads_started":1,"txn-participant-registration.queue_time_us":146}
W20250111 09:03:46.777767 25452 batcher.cc:441] Timed out: Failed to write batch of 1 ops to tablet e7a5d326eda64000b26dde512408aabb after 124 attempt(s): Failed to write to server: (no server available): Write(tablet: e7a5d326eda64000b26dde512408aabb, num_ops: 1, num_attempts: 124) passed its deadline: Illegal state: The tablet is not in a running state: SHUTDOWN
I20250111 09:03:46.783531 14580 tablet_server.cc:178] TabletServer@127.14.61.1:0 shutting down...
I20250111 09:03:46.788039 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250111 09:03:46.788187 14580 tablet_replica.cc:331] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06: stopping tablet replica
I20250111 09:03:46.788290 14580 raft_consensus.cc:2238] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250111 09:03:46.788368 14580 raft_consensus.cc:2267] T 778338e896ef4007aaf16fdc91644157 P c40510b56739488988af61e6f6774a06 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250111 09:03:46.794401 25368 meta_cache.cc:302] tablet e7a5d326eda64000b26dde512408aabb: replica c40510b56739488988af61e6f6774a06 (127.14.61.1:45703) has failed: Network error: Client connection negotiation failed: client connection to 127.14.61.1:45703: connect: Connection refused (error 111)
I20250111 09:03:46.800938 14580 tablet_server.cc:195] TabletServer@127.14.61.1:0 shutdown complete.
I20250111 09:03:46.803516 14580 tablet_server.cc:178] TabletServer@127.14.61.2:0 shutting down...
I20250111 09:03:46.807754 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250111 09:03:46.807898 14580 tablet_replica.cc:331] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479: stopping tablet replica
I20250111 09:03:46.807994 14580 raft_consensus.cc:2238] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:46.808166 14580 raft_consensus.cc:2267] T 778338e896ef4007aaf16fdc91644157 P 9c26b2b7505541018c43df669b6a8479 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:46.858569 14580 txn_status_manager.cc:765] Waiting for 1 task(s) to stop
I20250111 09:03:48.283507 25555 raft_consensus.cc:491] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c26b2b7505541018c43df669b6a8479)
I20250111 09:03:48.283641 25555 raft_consensus.cc:513] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } }
I20250111 09:03:48.283980 25555 leader_election.cc:290] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c40510b56739488988af61e6f6774a06 (127.14.61.1:45703), 9c26b2b7505541018c43df669b6a8479 (127.14.61.2:45241)
W20250111 09:03:48.284399 25379 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20250111 09:03:48.285038 25379 leader_election.cc:336] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c26b2b7505541018c43df669b6a8479 (127.14.61.2:45241): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20250111 09:03:48.285219 25382 leader_election.cc:336] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c40510b56739488988af61e6f6774a06 (127.14.61.1:45703): Network error: Client connection negotiation failed: client connection to 127.14.61.1:45703: connect: Connection refused (error 111)
I20250111 09:03:48.285279 25382 leader_election.cc:304] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8cac247555284a81a92a14ca81b24757; no voters: 9c26b2b7505541018c43df669b6a8479, c40510b56739488988af61e6f6774a06
I20250111 09:03:48.285403 25555 raft_consensus.cc:2743] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250111 09:03:50.271322 25558 raft_consensus.cc:491] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c26b2b7505541018c43df669b6a8479)
I20250111 09:03:50.271445 25558 raft_consensus.cc:513] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40510b56739488988af61e6f6774a06" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 45703 } } peers { permanent_uuid: "9c26b2b7505541018c43df669b6a8479" member_type: VOTER last_known_addr { host: "127.14.61.2" port: 45241 } } peers { permanent_uuid: "8cac247555284a81a92a14ca81b24757" member_type: VOTER last_known_addr { host: "127.14.61.3" port: 37641 } }
I20250111 09:03:50.271816 25558 leader_election.cc:290] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c40510b56739488988af61e6f6774a06 (127.14.61.1:45703), 9c26b2b7505541018c43df669b6a8479 (127.14.61.2:45241)
W20250111 09:03:50.272739 25379 leader_election.cc:336] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c26b2b7505541018c43df669b6a8479 (127.14.61.2:45241): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20250111 09:03:50.272955 25382 leader_election.cc:336] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c40510b56739488988af61e6f6774a06 (127.14.61.1:45703): Network error: Client connection negotiation failed: client connection to 127.14.61.1:45703: connect: Connection refused (error 111)
I20250111 09:03:50.273046 25382 leader_election.cc:304] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8cac247555284a81a92a14ca81b24757; no voters: 9c26b2b7505541018c43df669b6a8479, c40510b56739488988af61e6f6774a06
I20250111 09:03:50.273192 25558 raft_consensus.cc:2743] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250111 09:03:51.884321 14580 tablet_server.cc:195] TabletServer@127.14.61.2:0 shutdown complete.
I20250111 09:03:51.886612 14580 tablet_server.cc:178] TabletServer@127.14.61.3:0 shutting down...
I20250111 09:03:51.889724 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250111 09:03:51.889858 14580 tablet_replica.cc:331] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757: stopping tablet replica
I20250111 09:03:51.889953 14580 raft_consensus.cc:2238] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250111 09:03:51.890062 14580 raft_consensus.cc:2267] T 778338e896ef4007aaf16fdc91644157 P 8cac247555284a81a92a14ca81b24757 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:51.902022 14580 tablet_server.cc:195] TabletServer@127.14.61.3:0 shutdown complete.
I20250111 09:03:51.903808 14580 master.cc:537] Master@127.14.61.62:36387 shutting down...
I20250111 09:03:51.907282 14580 raft_consensus.cc:2238] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:51.907409 14580 raft_consensus.cc:2267] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:51.907477 14580 tablet_replica.cc:331] T 00000000000000000000000000000000 P 84392ea4d9734da0a8a14d6475ed2c0a: stopping tablet replica
I20250111 09:03:51.919544 14580 master.cc:559] Master@127.14.61.62:36387 shutdown complete.
[ OK ] TxnOpDispatcherITest.TxnWriteWhenReplicaIsShutdown (16388 ms)
[ RUN ] TxnOpDispatcherITest.TxnWriteOpHandlingEnabledFlag
I20250111 09:03:51.925729 14580 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.61.62:42941
I20250111 09:03:51.926028 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:51.927907 25561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:51.928469 14580 server_base.cc:1034] running on GCE node
W20250111 09:03:51.928963 25562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:51.929563 25564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:51.929709 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:51.929754 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:51.929772 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586231929772 us; error 0 us; skew 500 ppm
I20250111 09:03:51.930508 14580 webserver.cc:458] Webserver started at http://127.14.61.62:42729/ using document root <none> and password file <none>
I20250111 09:03:51.930631 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:51.930675 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:51.930738 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:51.931080 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteOpHandlingEnabledFlag.1736586070622223-14580-0/minicluster-data/master-0-root/instance:
uuid: "5eb13a02fa68408db9e63de725ba26e0"
format_stamp: "Formatted at 2025-01-11 09:03:51 on dist-test-slave-nvbt"
I20250111 09:03:51.932348 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:51.933070 25569 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:51.933260 14580 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250111 09:03:51.933318 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteOpHandlingEnabledFlag.1736586070622223-14580-0/minicluster-data/master-0-root
uuid: "5eb13a02fa68408db9e63de725ba26e0"
format_stamp: "Formatted at 2025-01-11 09:03:51 on dist-test-slave-nvbt"
I20250111 09:03:51.933377 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteOpHandlingEnabledFlag.1736586070622223-14580-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteOpHandlingEnabledFlag.1736586070622223-14580-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteOpHandlingEnabledFlag.1736586070622223-14580-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:51.946383 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:51.946671 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:51.951097 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:42941
I20250111 09:03:51.952777 25631 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:42941 every 8 connection(s)
I20250111 09:03:51.953260 25632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:51.954720 25632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0: Bootstrap starting.
I20250111 09:03:51.955404 25632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:51.956179 25632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0: No bootstrap required, opened a new log
I20250111 09:03:51.956547 25632 raft_consensus.cc:357] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb13a02fa68408db9e63de725ba26e0" member_type: VOTER }
I20250111 09:03:51.956624 25632 raft_consensus.cc:383] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:51.956657 25632 raft_consensus.cc:738] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eb13a02fa68408db9e63de725ba26e0, State: Initialized, Role: FOLLOWER
I20250111 09:03:51.956787 25632 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb13a02fa68408db9e63de725ba26e0" member_type: VOTER }
I20250111 09:03:51.956862 25632 raft_consensus.cc:397] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:51.956905 25632 raft_consensus.cc:491] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:51.956959 25632 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:51.957617 25632 raft_consensus.cc:513] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb13a02fa68408db9e63de725ba26e0" member_type: VOTER }
I20250111 09:03:51.957736 25632 leader_election.cc:304] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5eb13a02fa68408db9e63de725ba26e0; no voters:
I20250111 09:03:51.957871 25632 leader_election.cc:290] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:51.957954 25635 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:51.958112 25635 raft_consensus.cc:695] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0 [term 1 LEADER]: Becoming Leader. State: Replica: 5eb13a02fa68408db9e63de725ba26e0, State: Running, Role: LEADER
I20250111 09:03:51.958220 25632 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0 [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:03:51.958261 25635 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb13a02fa68408db9e63de725ba26e0" member_type: VOTER }
I20250111 09:03:51.958628 25636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5eb13a02fa68408db9e63de725ba26e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb13a02fa68408db9e63de725ba26e0" member_type: VOTER } }
I20250111 09:03:51.958663 25637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5eb13a02fa68408db9e63de725ba26e0. Latest consensus state: current_term: 1 leader_uuid: "5eb13a02fa68408db9e63de725ba26e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb13a02fa68408db9e63de725ba26e0" member_type: VOTER } }
I20250111 09:03:51.958760 25636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0 [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:51.958771 25637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0 [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:51.959235 25643 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:03:51.959873 25643 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:03:51.960038 14580 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250111 09:03:51.961527 25643 catalog_manager.cc:1348] Generated new cluster ID: 4d66c1ca588d498e9357d9741270dcdf
I20250111 09:03:51.961571 25643 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:03:51.972306 25643 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:03:51.972718 25643 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:03:51.978763 25643 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0: Generated new TSK 0
I20250111 09:03:51.978919 25643 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:03:51.989152 25578 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36534:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20250111 09:03:51.989806 25632 master.cc:440] Invalid argument: unable to initialize TxnManager: Error creating table kudu_system.kudu_transactions on the master: not enough live tablet servers to create a table with the requested replication factor 1; 0 tablet servers are alive: unable to init TxnManager, will retry in 1.000s
I20250111 09:03:51.992159 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:51.993815 25659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:51.994378 25660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:51.994979 25662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:51.995597 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:51.995734 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:51.995769 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:51.995783 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586231995783 us; error 0 us; skew 500 ppm
I20250111 09:03:51.996591 14580 webserver.cc:458] Webserver started at http://127.14.61.1:43053/ using document root <none> and password file <none>
I20250111 09:03:51.996757 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:51.996806 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:51.996863 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:51.997222 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteOpHandlingEnabledFlag.1736586070622223-14580-0/minicluster-data/ts-0-root/instance:
uuid: "48d3bbc90657406fa0a673a56bf3a886"
format_stamp: "Formatted at 2025-01-11 09:03:51 on dist-test-slave-nvbt"
I20250111 09:03:51.998478 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20250111 09:03:51.999207 25667 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:51.999441 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20250111 09:03:51.999504 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteOpHandlingEnabledFlag.1736586070622223-14580-0/minicluster-data/ts-0-root
uuid: "48d3bbc90657406fa0a673a56bf3a886"
format_stamp: "Formatted at 2025-01-11 09:03:51 on dist-test-slave-nvbt"
I20250111 09:03:51.999570 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteOpHandlingEnabledFlag.1736586070622223-14580-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteOpHandlingEnabledFlag.1736586070622223-14580-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TxnWriteOpHandlingEnabledFlag.1736586070622223-14580-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:52.008055 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:52.008297 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:52.008883 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:03:52.008934 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:52.009025 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:03:52.009100 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:52.013778 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:36455
I20250111 09:03:52.013804 25737 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:36455 every 8 connection(s)
I20250111 09:03:52.014648 25738 heartbeater.cc:346] Connected to a master server at 127.14.61.62:42941
I20250111 09:03:52.014729 25738 heartbeater.cc:463] Registering TS with master...
I20250111 09:03:52.014905 25738 heartbeater.cc:510] Master 127.14.61.62:42941 requested a full tablet report, sending...
I20250111 09:03:52.015413 25578 ts_manager.cc:194] Registered new tserver with Master: 48d3bbc90657406fa0a673a56bf3a886 (127.14.61.1:36455)
I20250111 09:03:52.016077 25578 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36550
I20250111 09:03:52.016369 14580 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002272422s
I20250111 09:03:52.021857 25578 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36574:
name: "txn_op_dispatcher_test"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
I20250111 09:03:52.028944 25697 tablet_service.cc:1467] Processing CreateTablet for tablet 603e9078a7554a70b385e15f486f4ab5 (DEFAULT_TABLE table=txn_op_dispatcher_test [id=330190868b594b9d8136b89e8d9b843d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:52.028988 25696 tablet_service.cc:1467] Processing CreateTablet for tablet 0508095e60be4fbf9b6a681d6dde7de0 (DEFAULT_TABLE table=txn_op_dispatcher_test [id=330190868b594b9d8136b89e8d9b843d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:52.029206 25697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 603e9078a7554a70b385e15f486f4ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:52.029323 25696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0508095e60be4fbf9b6a681d6dde7de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:52.030886 25748 tablet_bootstrap.cc:492] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886: Bootstrap starting.
I20250111 09:03:52.031585 25748 tablet_bootstrap.cc:654] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:52.032302 25748 tablet_bootstrap.cc:492] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886: No bootstrap required, opened a new log
I20250111 09:03:52.032362 25748 ts_tablet_manager.cc:1397] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:52.032622 25748 raft_consensus.cc:357] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d3bbc90657406fa0a673a56bf3a886" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36455 } }
I20250111 09:03:52.032692 25748 raft_consensus.cc:383] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:52.032717 25748 raft_consensus.cc:738] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48d3bbc90657406fa0a673a56bf3a886, State: Initialized, Role: FOLLOWER
I20250111 09:03:52.032802 25748 consensus_queue.cc:260] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d3bbc90657406fa0a673a56bf3a886" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36455 } }
I20250111 09:03:52.032864 25748 raft_consensus.cc:397] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:52.032891 25748 raft_consensus.cc:491] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:52.032920 25748 raft_consensus.cc:3054] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:52.033470 25748 raft_consensus.cc:513] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d3bbc90657406fa0a673a56bf3a886" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36455 } }
I20250111 09:03:52.033571 25748 leader_election.cc:304] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48d3bbc90657406fa0a673a56bf3a886; no voters:
I20250111 09:03:52.033691 25748 leader_election.cc:290] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:52.033799 25750 raft_consensus.cc:2798] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:52.033944 25748 ts_tablet_manager.cc:1428] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250111 09:03:52.033969 25750 raft_consensus.cc:695] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886 [term 1 LEADER]: Becoming Leader. State: Replica: 48d3bbc90657406fa0a673a56bf3a886, State: Running, Role: LEADER
I20250111 09:03:52.033980 25738 heartbeater.cc:502] Master 127.14.61.62:42941 was elected leader, sending a full tablet report...
I20250111 09:03:52.034148 25748 tablet_bootstrap.cc:492] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886: Bootstrap starting.
I20250111 09:03:52.034229 25750 consensus_queue.cc:237] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d3bbc90657406fa0a673a56bf3a886" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36455 } }
I20250111 09:03:52.034849 25748 tablet_bootstrap.cc:654] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:52.035498 25578 catalog_manager.cc:5526] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48d3bbc90657406fa0a673a56bf3a886 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "48d3bbc90657406fa0a673a56bf3a886" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d3bbc90657406fa0a673a56bf3a886" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36455 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:52.035605 25748 tablet_bootstrap.cc:492] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886: No bootstrap required, opened a new log
I20250111 09:03:52.035686 25748 ts_tablet_manager.cc:1397] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250111 09:03:52.035995 25748 raft_consensus.cc:357] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d3bbc90657406fa0a673a56bf3a886" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36455 } }
I20250111 09:03:52.036077 25748 raft_consensus.cc:383] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:52.036115 25748 raft_consensus.cc:738] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48d3bbc90657406fa0a673a56bf3a886, State: Initialized, Role: FOLLOWER
I20250111 09:03:52.036214 25748 consensus_queue.cc:260] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d3bbc90657406fa0a673a56bf3a886" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36455 } }
I20250111 09:03:52.036293 25748 raft_consensus.cc:397] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:52.036334 25748 raft_consensus.cc:491] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:52.036396 25748 raft_consensus.cc:3054] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:52.037207 25748 raft_consensus.cc:513] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d3bbc90657406fa0a673a56bf3a886" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36455 } }
I20250111 09:03:52.037330 25748 leader_election.cc:304] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48d3bbc90657406fa0a673a56bf3a886; no voters:
I20250111 09:03:52.037408 25748 leader_election.cc:290] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:52.037462 25750 raft_consensus.cc:2798] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:52.037560 25750 raft_consensus.cc:695] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886 [term 1 LEADER]: Becoming Leader. State: Replica: 48d3bbc90657406fa0a673a56bf3a886, State: Running, Role: LEADER
I20250111 09:03:52.037618 25748 ts_tablet_manager.cc:1428] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:52.037678 25750 consensus_queue.cc:237] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d3bbc90657406fa0a673a56bf3a886" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36455 } }
I20250111 09:03:52.038949 25578 catalog_manager.cc:5526] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48d3bbc90657406fa0a673a56bf3a886 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "48d3bbc90657406fa0a673a56bf3a886" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d3bbc90657406fa0a673a56bf3a886" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36455 } health_report { overall_health: HEALTHY } } }
W20250111 09:03:52.044142 25741 txn_manager_proxy_rpc.cc:150] re-attempting BeginTransaction request to one of TxnManagers
I20250111 09:03:52.997306 25578 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36578:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:53.001683 25697 tablet_service.cc:1467] Processing CreateTablet for tablet 49e7ea193a214efb9f3e409abf520af2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=527d28e62e4f437c8ebeb7c915c155f3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:03:53.001926 25697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49e7ea193a214efb9f3e409abf520af2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:53.003597 25766 tablet_bootstrap.cc:492] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886: Bootstrap starting.
I20250111 09:03:53.004302 25766 tablet_bootstrap.cc:654] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:53.005285 25766 tablet_bootstrap.cc:492] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886: No bootstrap required, opened a new log
I20250111 09:03:53.005357 25766 ts_tablet_manager.cc:1397] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:53.005649 25766 raft_consensus.cc:357] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d3bbc90657406fa0a673a56bf3a886" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36455 } }
I20250111 09:03:53.005731 25766 raft_consensus.cc:383] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:53.005765 25766 raft_consensus.cc:738] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48d3bbc90657406fa0a673a56bf3a886, State: Initialized, Role: FOLLOWER
I20250111 09:03:53.005864 25766 consensus_queue.cc:260] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d3bbc90657406fa0a673a56bf3a886" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36455 } }
I20250111 09:03:53.005944 25766 raft_consensus.cc:397] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:53.005975 25766 raft_consensus.cc:491] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:53.006011 25766 raft_consensus.cc:3054] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:53.006685 25766 raft_consensus.cc:513] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d3bbc90657406fa0a673a56bf3a886" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36455 } }
I20250111 09:03:53.006793 25766 leader_election.cc:304] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48d3bbc90657406fa0a673a56bf3a886; no voters:
I20250111 09:03:53.006940 25766 leader_election.cc:290] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:53.007040 25768 raft_consensus.cc:2798] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:53.007249 25768 raft_consensus.cc:695] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886 [term 1 LEADER]: Becoming Leader. State: Replica: 48d3bbc90657406fa0a673a56bf3a886, State: Running, Role: LEADER
I20250111 09:03:53.007292 25766 ts_tablet_manager.cc:1428] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250111 09:03:53.007385 25768 consensus_queue.cc:237] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d3bbc90657406fa0a673a56bf3a886" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36455 } }
I20250111 09:03:53.007819 25769 tablet_replica.cc:440] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48d3bbc90657406fa0a673a56bf3a886" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d3bbc90657406fa0a673a56bf3a886" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36455 } } }
I20250111 09:03:53.007928 25769 tablet_replica.cc:443] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:53.007926 25770 tablet_replica.cc:440] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886: TxnStatusTablet state changed. Reason: New leader 48d3bbc90657406fa0a673a56bf3a886. Latest consensus state: current_term: 1 leader_uuid: "48d3bbc90657406fa0a673a56bf3a886" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d3bbc90657406fa0a673a56bf3a886" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36455 } } }
I20250111 09:03:53.008051 25770 tablet_replica.cc:443] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:53.008211 25772 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:03:53.008299 25772 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:03:53.008661 25578 catalog_manager.cc:5526] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48d3bbc90657406fa0a673a56bf3a886 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "48d3bbc90657406fa0a673a56bf3a886" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d3bbc90657406fa0a673a56bf3a886" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 36455 } health_report { overall_health: HEALTHY } } }
W20250111 09:03:54.006734 25777 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:49072: Not found: txn 0 not found on tablet 603e9078a7554a70b385e15f486f4ab5
W20250111 09:03:54.007093 25741 batcher.cc:441] Not found: Failed to write batch of 1 ops to tablet 603e9078a7554a70b385e15f486f4ab5 after 1 attempt(s): Failed to write to server: 48d3bbc90657406fa0a673a56bf3a886 (127.14.61.1:36455): txn 0 not found on tablet 603e9078a7554a70b385e15f486f4ab5
I20250111 09:03:54.017112 14580 tablet_server.cc:178] TabletServer@127.14.61.1:0 shutting down...
I20250111 09:03:54.020352 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250111 09:03:54.020527 14580 tablet_replica.cc:331] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886: stopping tablet replica
I20250111 09:03:54.020627 14580 raft_consensus.cc:2238] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:54.020726 14580 raft_consensus.cc:2267] T 49e7ea193a214efb9f3e409abf520af2 P 48d3bbc90657406fa0a673a56bf3a886 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:54.021236 14580 tablet_replica.cc:331] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886: stopping tablet replica
I20250111 09:03:54.021318 14580 raft_consensus.cc:2238] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:54.021422 14580 raft_consensus.cc:2267] T 603e9078a7554a70b385e15f486f4ab5 P 48d3bbc90657406fa0a673a56bf3a886 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:54.021700 14580 tablet_replica.cc:331] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886: stopping tablet replica
I20250111 09:03:54.021767 14580 raft_consensus.cc:2238] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:54.021847 14580 raft_consensus.cc:2267] T 0508095e60be4fbf9b6a681d6dde7de0 P 48d3bbc90657406fa0a673a56bf3a886 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:54.034003 14580 tablet_server.cc:195] TabletServer@127.14.61.1:0 shutdown complete.
I20250111 09:03:54.035962 14580 master.cc:537] Master@127.14.61.62:42941 shutting down...
I20250111 09:03:54.039965 14580 raft_consensus.cc:2238] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:54.040068 14580 raft_consensus.cc:2267] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:54.040127 14580 tablet_replica.cc:331] T 00000000000000000000000000000000 P 5eb13a02fa68408db9e63de725ba26e0: stopping tablet replica
I20250111 09:03:54.051893 14580 master.cc:559] Master@127.14.61.62:42941 shutdown complete.
[ OK ] TxnOpDispatcherITest.TxnWriteOpHandlingEnabledFlag (2130 ms)
[ DISABLED ] TxnOpDispatcherITest.DISABLED_TxnMultipleSingleRowsWithServerRestart
[ RUN ] TxnOpDispatcherITest.TestBeginAbortTransactionalTestWorkload
I20250111 09:03:54.056480 14580 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.61.62:36825
I20250111 09:03:54.056767 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:54.058333 25783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:54.058862 25784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:54.059345 25786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:54.060007 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:54.060164 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:54.060195 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:54.060207 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586234060208 us; error 0 us; skew 500 ppm
I20250111 09:03:54.060956 14580 webserver.cc:458] Webserver started at http://127.14.61.62:43739/ using document root <none> and password file <none>
I20250111 09:03:54.061116 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:54.061157 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:54.061221 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:54.061549 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestBeginAbortTransactionalTestWorkload.1736586070622223-14580-0/minicluster-data/master-0-root/instance:
uuid: "f9d10c8fbee546a7818e23a5665e736e"
format_stamp: "Formatted at 2025-01-11 09:03:54 on dist-test-slave-nvbt"
I20250111 09:03:54.062707 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250111 09:03:54.063426 25791 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:54.063648 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:54.063701 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestBeginAbortTransactionalTestWorkload.1736586070622223-14580-0/minicluster-data/master-0-root
uuid: "f9d10c8fbee546a7818e23a5665e736e"
format_stamp: "Formatted at 2025-01-11 09:03:54 on dist-test-slave-nvbt"
I20250111 09:03:54.063747 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestBeginAbortTransactionalTestWorkload.1736586070622223-14580-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestBeginAbortTransactionalTestWorkload.1736586070622223-14580-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestBeginAbortTransactionalTestWorkload.1736586070622223-14580-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:54.073253 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:54.073479 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:54.077076 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:36825
I20250111 09:03:54.077378 25853 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:36825 every 8 connection(s)
I20250111 09:03:54.078155 25854 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:54.079506 25854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e: Bootstrap starting.
I20250111 09:03:54.080117 25854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:54.080839 25854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e: No bootstrap required, opened a new log
I20250111 09:03:54.081246 25854 raft_consensus.cc:357] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9d10c8fbee546a7818e23a5665e736e" member_type: VOTER }
I20250111 09:03:54.081323 25854 raft_consensus.cc:383] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:54.081354 25854 raft_consensus.cc:738] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d10c8fbee546a7818e23a5665e736e, State: Initialized, Role: FOLLOWER
I20250111 09:03:54.081465 25854 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9d10c8fbee546a7818e23a5665e736e" member_type: VOTER }
I20250111 09:03:54.081539 25854 raft_consensus.cc:397] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:54.081575 25854 raft_consensus.cc:491] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:54.081621 25854 raft_consensus.cc:3054] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:54.082145 25854 raft_consensus.cc:513] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9d10c8fbee546a7818e23a5665e736e" member_type: VOTER }
I20250111 09:03:54.082247 25854 leader_election.cc:304] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9d10c8fbee546a7818e23a5665e736e; no voters:
I20250111 09:03:54.082399 25854 leader_election.cc:290] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:54.082469 25857 raft_consensus.cc:2798] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:54.082636 25857 raft_consensus.cc:695] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e [term 1 LEADER]: Becoming Leader. State: Replica: f9d10c8fbee546a7818e23a5665e736e, State: Running, Role: LEADER
I20250111 09:03:54.082747 25854 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:03:54.082747 25857 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9d10c8fbee546a7818e23a5665e736e" member_type: VOTER }
I20250111 09:03:54.083129 25859 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9d10c8fbee546a7818e23a5665e736e. Latest consensus state: current_term: 1 leader_uuid: "f9d10c8fbee546a7818e23a5665e736e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9d10c8fbee546a7818e23a5665e736e" member_type: VOTER } }
I20250111 09:03:54.083223 25859 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:54.083110 25858 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9d10c8fbee546a7818e23a5665e736e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9d10c8fbee546a7818e23a5665e736e" member_type: VOTER } }
I20250111 09:03:54.083505 25863 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:03:54.083554 25858 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:54.084151 25863 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:03:54.084388 14580 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250111 09:03:54.085954 25863 catalog_manager.cc:1348] Generated new cluster ID: 506d4070711e466fa2602f9456a11ca3
I20250111 09:03:54.086009 25863 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:03:54.092871 25863 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:03:54.093302 25863 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:03:54.102039 25863 catalog_manager.cc:5899] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e: Generated new TSK 0
I20250111 09:03:54.102146 25863 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:03:54.110770 25808 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51454:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:54.116525 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:54.118211 25881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:54.118863 25882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:54.119271 25884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:54.119771 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:54.119896 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:54.119930 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:54.119943 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586234119943 us; error 0 us; skew 500 ppm
I20250111 09:03:54.120687 14580 webserver.cc:458] Webserver started at http://127.14.61.1:44843/ using document root <none> and password file <none>
I20250111 09:03:54.120815 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:54.120853 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:54.120936 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:54.121263 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestBeginAbortTransactionalTestWorkload.1736586070622223-14580-0/minicluster-data/ts-0-root/instance:
uuid: "62d89ce9cc6b44ad9df1c94d5aad621e"
format_stamp: "Formatted at 2025-01-11 09:03:54 on dist-test-slave-nvbt"
I20250111 09:03:54.122543 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20250111 09:03:54.123294 25889 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:54.123483 14580 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250111 09:03:54.123545 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestBeginAbortTransactionalTestWorkload.1736586070622223-14580-0/minicluster-data/ts-0-root
uuid: "62d89ce9cc6b44ad9df1c94d5aad621e"
format_stamp: "Formatted at 2025-01-11 09:03:54 on dist-test-slave-nvbt"
I20250111 09:03:54.123607 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestBeginAbortTransactionalTestWorkload.1736586070622223-14580-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestBeginAbortTransactionalTestWorkload.1736586070622223-14580-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestBeginAbortTransactionalTestWorkload.1736586070622223-14580-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:54.142939 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:54.143232 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:54.143872 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:03:54.143954 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:54.144008 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:03:54.144088 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:54.149173 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:32809
I20250111 09:03:54.149619 25959 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:32809 every 8 connection(s)
I20250111 09:03:54.149993 25960 heartbeater.cc:346] Connected to a master server at 127.14.61.62:36825
I20250111 09:03:54.150064 25960 heartbeater.cc:463] Registering TS with master...
I20250111 09:03:54.150244 25960 heartbeater.cc:510] Master 127.14.61.62:36825 requested a full tablet report, sending...
I20250111 09:03:54.150758 25808 ts_manager.cc:194] Registered new tserver with Master: 62d89ce9cc6b44ad9df1c94d5aad621e (127.14.61.1:32809)
I20250111 09:03:54.151484 25808 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51464
I20250111 09:03:54.151751 14580 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.00222853s
I20250111 09:03:54.152172 14580 test_util.cc:274] Using random seed: 2107360491
I20250111 09:03:54.158159 25807 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51494:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250111 09:03:54.165323 25919 tablet_service.cc:1467] Processing CreateTablet for tablet 813abd58c6ea4c80bf8564263599297f (DEFAULT_TABLE table=test-workload [id=3408d28abe81498f9c4013aa8d8035d2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:54.165403 25917 tablet_service.cc:1467] Processing CreateTablet for tablet 339ef2638a8f4f09828314433c01a3b1 (DEFAULT_TABLE table=test-workload [id=3408d28abe81498f9c4013aa8d8035d2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:54.165580 25919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 813abd58c6ea4c80bf8564263599297f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:54.165400 25918 tablet_service.cc:1467] Processing CreateTablet for tablet 5d5d1bf80a7a4dcfa3f4e8b70322475a (DEFAULT_TABLE table=test-workload [id=3408d28abe81498f9c4013aa8d8035d2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:54.165777 25917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 339ef2638a8f4f09828314433c01a3b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:54.165956 25918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d5d1bf80a7a4dcfa3f4e8b70322475a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:54.167882 25970 tablet_bootstrap.cc:492] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e: Bootstrap starting.
I20250111 09:03:54.168840 25970 tablet_bootstrap.cc:654] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:54.169745 25970 tablet_bootstrap.cc:492] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e: No bootstrap required, opened a new log
I20250111 09:03:54.169814 25970 ts_tablet_manager.cc:1397] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:54.170143 25970 raft_consensus.cc:357] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } }
I20250111 09:03:54.170220 25970 raft_consensus.cc:383] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:54.170277 25970 raft_consensus.cc:738] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62d89ce9cc6b44ad9df1c94d5aad621e, State: Initialized, Role: FOLLOWER
I20250111 09:03:54.170399 25970 consensus_queue.cc:260] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } }
I20250111 09:03:54.170462 25970 raft_consensus.cc:397] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:54.170514 25970 raft_consensus.cc:491] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:54.170562 25970 raft_consensus.cc:3054] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:54.171208 25970 raft_consensus.cc:513] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } }
I20250111 09:03:54.171330 25970 leader_election.cc:304] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62d89ce9cc6b44ad9df1c94d5aad621e; no voters:
I20250111 09:03:54.171483 25970 leader_election.cc:290] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:54.171638 25972 raft_consensus.cc:2798] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:54.171767 25970 ts_tablet_manager.cc:1428] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:54.171794 25960 heartbeater.cc:502] Master 127.14.61.62:36825 was elected leader, sending a full tablet report...
I20250111 09:03:54.171819 25972 raft_consensus.cc:695] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e [term 1 LEADER]: Becoming Leader. State: Replica: 62d89ce9cc6b44ad9df1c94d5aad621e, State: Running, Role: LEADER
I20250111 09:03:54.171928 25970 tablet_bootstrap.cc:492] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e: Bootstrap starting.
I20250111 09:03:54.171921 25972 consensus_queue.cc:237] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } }
I20250111 09:03:54.172768 25970 tablet_bootstrap.cc:654] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:54.173190 25807 catalog_manager.cc:5526] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e reported cstate change: term changed from 0 to 1, leader changed from <none> to 62d89ce9cc6b44ad9df1c94d5aad621e (127.14.61.1). New cstate: current_term: 1 leader_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:54.173768 25970 tablet_bootstrap.cc:492] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e: No bootstrap required, opened a new log
I20250111 09:03:54.173827 25970 ts_tablet_manager.cc:1397] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250111 09:03:54.174165 25970 raft_consensus.cc:357] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } }
I20250111 09:03:54.174239 25970 raft_consensus.cc:383] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:54.174261 25970 raft_consensus.cc:738] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62d89ce9cc6b44ad9df1c94d5aad621e, State: Initialized, Role: FOLLOWER
I20250111 09:03:54.174373 25970 consensus_queue.cc:260] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } }
I20250111 09:03:54.174450 25970 raft_consensus.cc:397] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:54.174506 25970 raft_consensus.cc:491] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:54.174556 25970 raft_consensus.cc:3054] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:54.175354 25970 raft_consensus.cc:513] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } }
I20250111 09:03:54.175468 25970 leader_election.cc:304] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62d89ce9cc6b44ad9df1c94d5aad621e; no voters:
I20250111 09:03:54.175557 25970 leader_election.cc:290] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:54.175599 25972 raft_consensus.cc:2798] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:54.175698 25972 raft_consensus.cc:695] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e [term 1 LEADER]: Becoming Leader. State: Replica: 62d89ce9cc6b44ad9df1c94d5aad621e, State: Running, Role: LEADER
I20250111 09:03:54.175765 25970 ts_tablet_manager.cc:1428] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250111 09:03:54.175786 25972 consensus_queue.cc:237] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } }
I20250111 09:03:54.175874 25970 tablet_bootstrap.cc:492] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e: Bootstrap starting.
I20250111 09:03:54.176599 25970 tablet_bootstrap.cc:654] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:54.176959 25807 catalog_manager.cc:5526] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e reported cstate change: term changed from 0 to 1, leader changed from <none> to 62d89ce9cc6b44ad9df1c94d5aad621e (127.14.61.1). New cstate: current_term: 1 leader_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:54.177594 25970 tablet_bootstrap.cc:492] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e: No bootstrap required, opened a new log
I20250111 09:03:54.177681 25970 ts_tablet_manager.cc:1397] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250111 09:03:54.178046 25970 raft_consensus.cc:357] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } }
I20250111 09:03:54.178125 25970 raft_consensus.cc:383] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:54.178156 25970 raft_consensus.cc:738] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62d89ce9cc6b44ad9df1c94d5aad621e, State: Initialized, Role: FOLLOWER
I20250111 09:03:54.178244 25970 consensus_queue.cc:260] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } }
I20250111 09:03:54.178313 25970 raft_consensus.cc:397] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:54.178351 25970 raft_consensus.cc:491] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:54.178393 25970 raft_consensus.cc:3054] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:54.179013 25970 raft_consensus.cc:513] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } }
I20250111 09:03:54.179103 25970 leader_election.cc:304] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62d89ce9cc6b44ad9df1c94d5aad621e; no voters:
I20250111 09:03:54.179196 25970 leader_election.cc:290] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:54.179245 25972 raft_consensus.cc:2798] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:54.179417 25970 ts_tablet_manager.cc:1428] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250111 09:03:54.179503 25972 raft_consensus.cc:695] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 1 LEADER]: Becoming Leader. State: Replica: 62d89ce9cc6b44ad9df1c94d5aad621e, State: Running, Role: LEADER
I20250111 09:03:54.179626 25972 consensus_queue.cc:237] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } }
I20250111 09:03:54.180620 25807 catalog_manager.cc:5526] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e reported cstate change: term changed from 0 to 1, leader changed from <none> to 62d89ce9cc6b44ad9df1c94d5aad621e (127.14.61.1). New cstate: current_term: 1 leader_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } health_report { overall_health: HEALTHY } } }
W20250111 09:03:54.185796 25966 txn_manager_proxy_rpc.cc:150] re-attempting BeginTransaction request to one of TxnManagers
I20250111 09:03:55.118840 25807 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52948:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:55.123346 25919 tablet_service.cc:1467] Processing CreateTablet for tablet 76d213c767ef48758ac7c044eb41cf0f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=62b33803865b4ae7a682e542dabafa68]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:03:55.123605 25919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76d213c767ef48758ac7c044eb41cf0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:55.125613 25990 tablet_bootstrap.cc:492] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e: Bootstrap starting.
I20250111 09:03:55.126505 25990 tablet_bootstrap.cc:654] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:55.127383 25990 tablet_bootstrap.cc:492] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e: No bootstrap required, opened a new log
I20250111 09:03:55.127454 25990 ts_tablet_manager.cc:1397] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:55.127750 25990 raft_consensus.cc:357] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } }
I20250111 09:03:55.127827 25990 raft_consensus.cc:383] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:55.127868 25990 raft_consensus.cc:738] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62d89ce9cc6b44ad9df1c94d5aad621e, State: Initialized, Role: FOLLOWER
I20250111 09:03:55.127950 25990 consensus_queue.cc:260] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } }
I20250111 09:03:55.128005 25990 raft_consensus.cc:397] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:55.128032 25990 raft_consensus.cc:491] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:55.128077 25990 raft_consensus.cc:3054] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:55.128674 25990 raft_consensus.cc:513] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } }
I20250111 09:03:55.128775 25990 leader_election.cc:304] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62d89ce9cc6b44ad9df1c94d5aad621e; no voters:
I20250111 09:03:55.128949 25990 leader_election.cc:290] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:55.129041 25992 raft_consensus.cc:2798] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:55.129281 25992 raft_consensus.cc:695] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 1 LEADER]: Becoming Leader. State: Replica: 62d89ce9cc6b44ad9df1c94d5aad621e, State: Running, Role: LEADER
I20250111 09:03:55.129343 25990 ts_tablet_manager.cc:1428] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:55.129607 25992 consensus_queue.cc:237] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } }
I20250111 09:03:55.129956 25994 tablet_replica.cc:440] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } } }
I20250111 09:03:55.130070 25994 tablet_replica.cc:443] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:55.130007 25993 tablet_replica.cc:440] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e: TxnStatusTablet state changed. Reason: New leader 62d89ce9cc6b44ad9df1c94d5aad621e. Latest consensus state: current_term: 1 leader_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } } }
I20250111 09:03:55.130208 25993 tablet_replica.cc:443] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:55.130523 25996 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:03:55.130712 25996 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:03:55.130821 25807 catalog_manager.cc:5526] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e reported cstate change: term changed from 0 to 1, leader changed from <none> to 62d89ce9cc6b44ad9df1c94d5aad621e (127.14.61.1). New cstate: current_term: 1 leader_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d89ce9cc6b44ad9df1c94d5aad621e" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 32809 } health_report { overall_health: HEALTHY } } }
W20250111 09:03:55.752326 26001 tablet_replica.cc:1307] Aborted: operation has been aborted: cancelling pending write operations
I20250111 09:03:55.759991 14580 tablet_server.cc:178] TabletServer@127.14.61.1:0 shutting down...
I20250111 09:03:55.763437 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250111 09:03:55.763841 14580 tablet_replica.cc:331] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e: stopping tablet replica
I20250111 09:03:55.763968 14580 raft_consensus.cc:2238] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:55.764083 14580 raft_consensus.cc:2267] T 76d213c767ef48758ac7c044eb41cf0f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:55.764582 14580 tablet_replica.cc:331] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e: stopping tablet replica
I20250111 09:03:55.764671 14580 raft_consensus.cc:2238] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:55.764760 14580 raft_consensus.cc:2267] T 339ef2638a8f4f09828314433c01a3b1 P 62d89ce9cc6b44ad9df1c94d5aad621e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:55.765229 14580 tablet_replica.cc:331] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e: stopping tablet replica
I20250111 09:03:55.765331 14580 raft_consensus.cc:2238] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:55.765426 14580 raft_consensus.cc:2267] T 5d5d1bf80a7a4dcfa3f4e8b70322475a P 62d89ce9cc6b44ad9df1c94d5aad621e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:55.765856 14580 tablet_replica.cc:331] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e: stopping tablet replica
I20250111 09:03:55.765944 14580 raft_consensus.cc:2238] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:55.766041 14580 raft_consensus.cc:2267] T 813abd58c6ea4c80bf8564263599297f P 62d89ce9cc6b44ad9df1c94d5aad621e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:55.779232 14580 tablet_server.cc:195] TabletServer@127.14.61.1:0 shutdown complete.
I20250111 09:03:55.781818 14580 master.cc:537] Master@127.14.61.62:36825 shutting down...
I20250111 09:03:55.786516 14580 raft_consensus.cc:2238] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:55.786675 14580 raft_consensus.cc:2267] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:55.786789 14580 tablet_replica.cc:331] T 00000000000000000000000000000000 P f9d10c8fbee546a7818e23a5665e736e: stopping tablet replica
I20250111 09:03:55.798787 14580 master.cc:559] Master@127.14.61.62:36825 shutdown complete.
[ OK ] TxnOpDispatcherITest.TestBeginAbortTransactionalTestWorkload (1747 ms)
[ RUN ] TxnOpDispatcherITest.TestBeginCommitTransactionalTestWorkload
I20250111 09:03:55.804014 14580 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.61.62:44583
I20250111 09:03:55.804325 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:55.806233 26030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:55.806941 26028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:55.807169 26027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:55.807569 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:55.807714 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:55.807752 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:55.807773 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586235807773 us; error 0 us; skew 500 ppm
I20250111 09:03:55.808615 14580 webserver.cc:458] Webserver started at http://127.14.61.62:46049/ using document root <none> and password file <none>
I20250111 09:03:55.808737 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:55.808779 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:55.808843 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:55.809223 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestBeginCommitTransactionalTestWorkload.1736586070622223-14580-0/minicluster-data/master-0-root/instance:
uuid: "d0816985e4c043648de3c72372679d29"
format_stamp: "Formatted at 2025-01-11 09:03:55 on dist-test-slave-nvbt"
I20250111 09:03:55.810429 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:55.811250 26035 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:55.811520 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:55.811578 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestBeginCommitTransactionalTestWorkload.1736586070622223-14580-0/minicluster-data/master-0-root
uuid: "d0816985e4c043648de3c72372679d29"
format_stamp: "Formatted at 2025-01-11 09:03:55 on dist-test-slave-nvbt"
I20250111 09:03:55.811645 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestBeginCommitTransactionalTestWorkload.1736586070622223-14580-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestBeginCommitTransactionalTestWorkload.1736586070622223-14580-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestBeginCommitTransactionalTestWorkload.1736586070622223-14580-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:55.825979 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:55.826243 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:55.831175 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:44583
I20250111 09:03:55.832276 26097 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:44583 every 8 connection(s)
I20250111 09:03:55.832662 26098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:55.834214 26098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29: Bootstrap starting.
I20250111 09:03:55.834856 26098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:55.835678 26098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29: No bootstrap required, opened a new log
I20250111 09:03:55.836014 26098 raft_consensus.cc:357] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0816985e4c043648de3c72372679d29" member_type: VOTER }
I20250111 09:03:55.836088 26098 raft_consensus.cc:383] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:55.836109 26098 raft_consensus.cc:738] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0816985e4c043648de3c72372679d29, State: Initialized, Role: FOLLOWER
I20250111 09:03:55.836278 26098 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0816985e4c043648de3c72372679d29" member_type: VOTER }
I20250111 09:03:55.836341 26098 raft_consensus.cc:397] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:55.836376 26098 raft_consensus.cc:491] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:55.836422 26098 raft_consensus.cc:3054] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:55.837065 26098 raft_consensus.cc:513] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0816985e4c043648de3c72372679d29" member_type: VOTER }
I20250111 09:03:55.837172 26098 leader_election.cc:304] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0816985e4c043648de3c72372679d29; no voters:
I20250111 09:03:55.837356 26098 leader_election.cc:290] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:55.837461 26101 raft_consensus.cc:2798] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:55.837700 26101 raft_consensus.cc:695] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29 [term 1 LEADER]: Becoming Leader. State: Replica: d0816985e4c043648de3c72372679d29, State: Running, Role: LEADER
I20250111 09:03:55.837740 26098 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29 [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:03:55.837894 26101 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0816985e4c043648de3c72372679d29" member_type: VOTER }
I20250111 09:03:55.838238 26102 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0816985e4c043648de3c72372679d29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0816985e4c043648de3c72372679d29" member_type: VOTER } }
I20250111 09:03:55.838307 26102 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29 [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:55.838572 26103 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0816985e4c043648de3c72372679d29. Latest consensus state: current_term: 1 leader_uuid: "d0816985e4c043648de3c72372679d29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0816985e4c043648de3c72372679d29" member_type: VOTER } }
I20250111 09:03:55.838692 26103 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29 [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:55.838873 26106 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:03:55.839692 26106 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:03:55.839862 14580 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250111 09:03:55.845568 26106 catalog_manager.cc:1348] Generated new cluster ID: 822b2463494e4518be9a2b1d3a182fa0
I20250111 09:03:55.845620 26106 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:03:55.858875 26106 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:03:55.859484 26106 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:03:55.868593 26106 catalog_manager.cc:5899] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29: Generated new TSK 0
I20250111 09:03:55.868719 26106 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:03:55.872074 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250111 09:03:55.873873 26052 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60200:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20250111 09:03:55.874652 26128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:55.874758 26125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:55.875418 26126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:55.876040 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:55.876188 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:55.876236 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:55.876256 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586235876256 us; error 0 us; skew 500 ppm
I20250111 09:03:55.877189 14580 webserver.cc:458] Webserver started at http://127.14.61.1:36389/ using document root <none> and password file <none>
I20250111 09:03:55.877360 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:55.877420 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:55.877501 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:55.877941 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestBeginCommitTransactionalTestWorkload.1736586070622223-14580-0/minicluster-data/ts-0-root/instance:
uuid: "b1518c13f39e4788a058c8a3a0bd7517"
format_stamp: "Formatted at 2025-01-11 09:03:55 on dist-test-slave-nvbt"
I20250111 09:03:55.879601 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20250111 09:03:55.880558 26133 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:55.880801 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250111 09:03:55.880854 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestBeginCommitTransactionalTestWorkload.1736586070622223-14580-0/minicluster-data/ts-0-root
uuid: "b1518c13f39e4788a058c8a3a0bd7517"
format_stamp: "Formatted at 2025-01-11 09:03:55 on dist-test-slave-nvbt"
I20250111 09:03:55.880908 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestBeginCommitTransactionalTestWorkload.1736586070622223-14580-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestBeginCommitTransactionalTestWorkload.1736586070622223-14580-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestBeginCommitTransactionalTestWorkload.1736586070622223-14580-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:55.889205 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:55.889463 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:55.890148 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:03:55.890225 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:55.890302 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:03:55.890359 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:55.895037 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:39149
I20250111 09:03:55.897599 26202 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:39149 every 8 connection(s)
I20250111 09:03:55.898316 26203 heartbeater.cc:346] Connected to a master server at 127.14.61.62:44583
I20250111 09:03:55.898419 26203 heartbeater.cc:463] Registering TS with master...
I20250111 09:03:55.898579 26203 heartbeater.cc:510] Master 127.14.61.62:44583 requested a full tablet report, sending...
I20250111 09:03:55.899062 26052 ts_manager.cc:194] Registered new tserver with Master: b1518c13f39e4788a058c8a3a0bd7517 (127.14.61.1:39149)
I20250111 09:03:55.900017 14580 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.004535098s
I20250111 09:03:55.900074 26052 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60210
I20250111 09:03:55.900527 14580 test_util.cc:274] Using random seed: 2109108845
I20250111 09:03:55.906983 26052 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60226:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250111 09:03:55.914990 26161 tablet_service.cc:1467] Processing CreateTablet for tablet c6c38b1896e14682bde9a19b28f55ce7 (DEFAULT_TABLE table=test-workload [id=969956b280774902942f6eb1095d5af9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:55.915045 26157 tablet_service.cc:1467] Processing CreateTablet for tablet 48754a41cf9e447f84e7ca5d30da1570 (DEFAULT_TABLE table=test-workload [id=969956b280774902942f6eb1095d5af9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:55.915045 26163 tablet_service.cc:1467] Processing CreateTablet for tablet 9dacb8d685e241c28f4a6e12f6e6798f (DEFAULT_TABLE table=test-workload [id=969956b280774902942f6eb1095d5af9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:55.915252 26161 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6c38b1896e14682bde9a19b28f55ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:55.915374 26163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dacb8d685e241c28f4a6e12f6e6798f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:55.915537 26157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48754a41cf9e447f84e7ca5d30da1570. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:55.917680 26214 tablet_bootstrap.cc:492] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517: Bootstrap starting.
I20250111 09:03:55.918535 26214 tablet_bootstrap.cc:654] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:55.919468 26214 tablet_bootstrap.cc:492] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517: No bootstrap required, opened a new log
I20250111 09:03:55.919554 26214 ts_tablet_manager.cc:1397] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:55.919957 26214 raft_consensus.cc:357] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } }
I20250111 09:03:55.920050 26214 raft_consensus.cc:383] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:55.920099 26214 raft_consensus.cc:738] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1518c13f39e4788a058c8a3a0bd7517, State: Initialized, Role: FOLLOWER
I20250111 09:03:55.920207 26214 consensus_queue.cc:260] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } }
I20250111 09:03:55.920277 26214 raft_consensus.cc:397] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:55.920331 26214 raft_consensus.cc:491] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:55.920372 26214 raft_consensus.cc:3054] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:55.921166 26214 raft_consensus.cc:513] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } }
I20250111 09:03:55.921350 26214 leader_election.cc:304] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1518c13f39e4788a058c8a3a0bd7517; no voters:
I20250111 09:03:55.921543 26214 leader_election.cc:290] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:55.921619 26216 raft_consensus.cc:2798] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:55.921788 26214 ts_tablet_manager.cc:1428] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250111 09:03:55.921787 26216 raft_consensus.cc:695] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517 [term 1 LEADER]: Becoming Leader. State: Replica: b1518c13f39e4788a058c8a3a0bd7517, State: Running, Role: LEADER
I20250111 09:03:55.921959 26203 heartbeater.cc:502] Master 127.14.61.62:44583 was elected leader, sending a full tablet report...
I20250111 09:03:55.921984 26214 tablet_bootstrap.cc:492] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517: Bootstrap starting.
I20250111 09:03:55.922086 26216 consensus_queue.cc:237] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } }
I20250111 09:03:55.923038 26214 tablet_bootstrap.cc:654] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:55.923393 26052 catalog_manager.cc:5526] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1518c13f39e4788a058c8a3a0bd7517 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "b1518c13f39e4788a058c8a3a0bd7517" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:55.924077 26214 tablet_bootstrap.cc:492] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517: No bootstrap required, opened a new log
I20250111 09:03:55.924156 26214 ts_tablet_manager.cc:1397] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250111 09:03:55.924568 26214 raft_consensus.cc:357] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } }
I20250111 09:03:55.924650 26214 raft_consensus.cc:383] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:55.924683 26214 raft_consensus.cc:738] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1518c13f39e4788a058c8a3a0bd7517, State: Initialized, Role: FOLLOWER
I20250111 09:03:55.924774 26214 consensus_queue.cc:260] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } }
I20250111 09:03:55.924839 26214 raft_consensus.cc:397] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:55.924860 26214 raft_consensus.cc:491] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:55.924885 26214 raft_consensus.cc:3054] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:55.925555 26214 raft_consensus.cc:513] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } }
I20250111 09:03:55.925661 26214 leader_election.cc:304] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1518c13f39e4788a058c8a3a0bd7517; no voters:
I20250111 09:03:55.925742 26214 leader_election.cc:290] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:55.925873 26217 raft_consensus.cc:2798] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:55.925961 26217 raft_consensus.cc:695] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517 [term 1 LEADER]: Becoming Leader. State: Replica: b1518c13f39e4788a058c8a3a0bd7517, State: Running, Role: LEADER
I20250111 09:03:55.926074 26217 consensus_queue.cc:237] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } }
I20250111 09:03:55.926545 26214 ts_tablet_manager.cc:1428] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20250111 09:03:55.926740 26214 tablet_bootstrap.cc:492] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517: Bootstrap starting.
I20250111 09:03:55.927202 26051 catalog_manager.cc:5526] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1518c13f39e4788a058c8a3a0bd7517 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "b1518c13f39e4788a058c8a3a0bd7517" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:55.927490 26214 tablet_bootstrap.cc:654] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:55.928418 26214 tablet_bootstrap.cc:492] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517: No bootstrap required, opened a new log
I20250111 09:03:55.928493 26214 ts_tablet_manager.cc:1397] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:55.928867 26214 raft_consensus.cc:357] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } }
I20250111 09:03:55.928953 26214 raft_consensus.cc:383] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:55.929037 26214 raft_consensus.cc:738] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1518c13f39e4788a058c8a3a0bd7517, State: Initialized, Role: FOLLOWER
I20250111 09:03:55.929157 26214 consensus_queue.cc:260] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } }
I20250111 09:03:55.929231 26214 raft_consensus.cc:397] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:55.929265 26214 raft_consensus.cc:491] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:55.929299 26214 raft_consensus.cc:3054] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:55.929996 26214 raft_consensus.cc:513] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } }
I20250111 09:03:55.930114 26214 leader_election.cc:304] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1518c13f39e4788a058c8a3a0bd7517; no voters:
I20250111 09:03:55.930243 26217 raft_consensus.cc:2798] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:55.930326 26217 raft_consensus.cc:695] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517 [term 1 LEADER]: Becoming Leader. State: Replica: b1518c13f39e4788a058c8a3a0bd7517, State: Running, Role: LEADER
I20250111 09:03:55.930430 26217 consensus_queue.cc:237] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } }
I20250111 09:03:55.930740 26214 leader_election.cc:290] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:55.931041 26214 ts_tablet_manager.cc:1428] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:55.931424 26051 catalog_manager.cc:5526] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1518c13f39e4788a058c8a3a0bd7517 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "b1518c13f39e4788a058c8a3a0bd7517" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } health_report { overall_health: HEALTHY } } }
W20250111 09:03:55.936278 26208 txn_manager_proxy_rpc.cc:150] re-attempting BeginTransaction request to one of TxnManagers
I20250111 09:03:56.887995 26051 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60242:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:56.892812 26157 tablet_service.cc:1467] Processing CreateTablet for tablet 38f4340ca98e426c91bcd951abe0911e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=335c9f2b4fc7408d98e572fe7fc90e55]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:03:56.893095 26157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f4340ca98e426c91bcd951abe0911e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:56.894723 26234 tablet_bootstrap.cc:492] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517: Bootstrap starting.
I20250111 09:03:56.895534 26234 tablet_bootstrap.cc:654] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:56.896356 26234 tablet_bootstrap.cc:492] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517: No bootstrap required, opened a new log
I20250111 09:03:56.896421 26234 ts_tablet_manager.cc:1397] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:56.896688 26234 raft_consensus.cc:357] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } }
I20250111 09:03:56.896759 26234 raft_consensus.cc:383] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:56.896786 26234 raft_consensus.cc:738] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1518c13f39e4788a058c8a3a0bd7517, State: Initialized, Role: FOLLOWER
I20250111 09:03:56.896903 26234 consensus_queue.cc:260] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } }
I20250111 09:03:56.896960 26234 raft_consensus.cc:397] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:56.896989 26234 raft_consensus.cc:491] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:56.897061 26234 raft_consensus.cc:3054] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:56.897717 26234 raft_consensus.cc:513] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } }
I20250111 09:03:56.897825 26234 leader_election.cc:304] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1518c13f39e4788a058c8a3a0bd7517; no voters:
I20250111 09:03:56.898022 26234 leader_election.cc:290] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:56.898116 26236 raft_consensus.cc:2798] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:56.898278 26236 raft_consensus.cc:695] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517 [term 1 LEADER]: Becoming Leader. State: Replica: b1518c13f39e4788a058c8a3a0bd7517, State: Running, Role: LEADER
I20250111 09:03:56.898355 26234 ts_tablet_manager.cc:1428] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:56.898631 26236 consensus_queue.cc:237] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } }
I20250111 09:03:56.899101 26237 tablet_replica.cc:440] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517: TxnStatusTablet state changed. Reason: New leader b1518c13f39e4788a058c8a3a0bd7517. Latest consensus state: current_term: 1 leader_uuid: "b1518c13f39e4788a058c8a3a0bd7517" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } } }
I20250111 09:03:56.899194 26237 tablet_replica.cc:443] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:56.899634 26240 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:03:56.899725 26240 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:03:56.899726 26238 tablet_replica.cc:440] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1518c13f39e4788a058c8a3a0bd7517" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } } }
I20250111 09:03:56.899819 26238 tablet_replica.cc:443] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:56.899909 26051 catalog_manager.cc:5526] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1518c13f39e4788a058c8a3a0bd7517 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "b1518c13f39e4788a058c8a3a0bd7517" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1518c13f39e4788a058c8a3a0bd7517" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 39149 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:57.098603 14580 tablet_server.cc:178] TabletServer@127.14.61.1:0 shutting down...
I20250111 09:03:57.102090 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250111 09:03:57.102485 14580 tablet_replica.cc:331] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517: stopping tablet replica
I20250111 09:03:57.102594 14580 raft_consensus.cc:2238] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:57.102677 14580 raft_consensus.cc:2267] T 38f4340ca98e426c91bcd951abe0911e P b1518c13f39e4788a058c8a3a0bd7517 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:57.103164 14580 tablet_replica.cc:331] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517: stopping tablet replica
I20250111 09:03:57.103235 14580 raft_consensus.cc:2238] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:57.103307 14580 raft_consensus.cc:2267] T 48754a41cf9e447f84e7ca5d30da1570 P b1518c13f39e4788a058c8a3a0bd7517 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:57.103664 14580 tablet_replica.cc:331] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517: stopping tablet replica
I20250111 09:03:57.103727 14580 raft_consensus.cc:2238] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:57.103791 14580 raft_consensus.cc:2267] T c6c38b1896e14682bde9a19b28f55ce7 P b1518c13f39e4788a058c8a3a0bd7517 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:57.104218 14580 tablet_replica.cc:331] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517: stopping tablet replica
I20250111 09:03:57.104282 14580 raft_consensus.cc:2238] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:57.104343 14580 raft_consensus.cc:2267] T 9dacb8d685e241c28f4a6e12f6e6798f P b1518c13f39e4788a058c8a3a0bd7517 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:57.107313 14580 tablet_server.cc:195] TabletServer@127.14.61.1:0 shutdown complete.
I20250111 09:03:57.109630 14580 master.cc:537] Master@127.14.61.62:44583 shutting down...
I20250111 09:03:57.113247 14580 raft_consensus.cc:2238] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:57.113350 14580 raft_consensus.cc:2267] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:57.113404 14580 tablet_replica.cc:331] T 00000000000000000000000000000000 P d0816985e4c043648de3c72372679d29: stopping tablet replica
I20250111 09:03:57.125589 14580 master.cc:559] Master@127.14.61.62:44583 shutdown complete.
[ OK ] TxnOpDispatcherITest.TestBeginCommitTransactionalTestWorkload (1327 ms)
[ RUN ] TxnOpDispatcherITest.TestSeparateBeginCommitTestWorkloads
I20250111 09:03:57.131057 14580 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.61.62:42775
I20250111 09:03:57.131402 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:57.133147 26265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:57.133745 26268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:57.134383 26266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:57.134970 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:57.135097 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:57.135133 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:57.135151 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586237135150 us; error 0 us; skew 500 ppm
I20250111 09:03:57.135985 14580 webserver.cc:458] Webserver started at http://127.14.61.62:38469/ using document root <none> and password file <none>
I20250111 09:03:57.136135 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:57.136178 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:57.136258 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:57.136610 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestSeparateBeginCommitTestWorkloads.1736586070622223-14580-0/minicluster-data/master-0-root/instance:
uuid: "3137585be172470ea2a6689a1be1a866"
format_stamp: "Formatted at 2025-01-11 09:03:57 on dist-test-slave-nvbt"
I20250111 09:03:57.138113 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250111 09:03:57.138839 26273 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:57.139070 14580 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20250111 09:03:57.139127 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestSeparateBeginCommitTestWorkloads.1736586070622223-14580-0/minicluster-data/master-0-root
uuid: "3137585be172470ea2a6689a1be1a866"
format_stamp: "Formatted at 2025-01-11 09:03:57 on dist-test-slave-nvbt"
I20250111 09:03:57.139194 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestSeparateBeginCommitTestWorkloads.1736586070622223-14580-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestSeparateBeginCommitTestWorkloads.1736586070622223-14580-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestSeparateBeginCommitTestWorkloads.1736586070622223-14580-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:57.144254 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:57.144498 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:57.148499 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.62:42775
I20250111 09:03:57.149902 26335 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.62:42775 every 8 connection(s)
I20250111 09:03:57.150269 26336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:57.151793 26336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866: Bootstrap starting.
I20250111 09:03:57.152472 26336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:57.153267 26336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866: No bootstrap required, opened a new log
I20250111 09:03:57.153563 26336 raft_consensus.cc:357] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3137585be172470ea2a6689a1be1a866" member_type: VOTER }
I20250111 09:03:57.153635 26336 raft_consensus.cc:383] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:57.153662 26336 raft_consensus.cc:738] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3137585be172470ea2a6689a1be1a866, State: Initialized, Role: FOLLOWER
I20250111 09:03:57.153750 26336 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3137585be172470ea2a6689a1be1a866" member_type: VOTER }
I20250111 09:03:57.153800 26336 raft_consensus.cc:397] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:57.153827 26336 raft_consensus.cc:491] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:57.153861 26336 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:57.154409 26336 raft_consensus.cc:513] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3137585be172470ea2a6689a1be1a866" member_type: VOTER }
I20250111 09:03:57.154508 26336 leader_election.cc:304] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3137585be172470ea2a6689a1be1a866; no voters:
I20250111 09:03:57.154639 26336 leader_election.cc:290] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:57.154739 26339 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:57.154889 26339 raft_consensus.cc:695] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866 [term 1 LEADER]: Becoming Leader. State: Replica: 3137585be172470ea2a6689a1be1a866, State: Running, Role: LEADER
I20250111 09:03:57.155071 26336 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866 [sys.catalog]: configured and running, proceeding with master startup.
I20250111 09:03:57.155066 26339 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3137585be172470ea2a6689a1be1a866" member_type: VOTER }
I20250111 09:03:57.155519 26340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3137585be172470ea2a6689a1be1a866" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3137585be172470ea2a6689a1be1a866" member_type: VOTER } }
I20250111 09:03:57.155563 26341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3137585be172470ea2a6689a1be1a866. Latest consensus state: current_term: 1 leader_uuid: "3137585be172470ea2a6689a1be1a866" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3137585be172470ea2a6689a1be1a866" member_type: VOTER } }
I20250111 09:03:57.155761 26341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866 [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:57.155671 26340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866 [sys.catalog]: This master's current role is: LEADER
I20250111 09:03:57.156322 26349 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 09:03:57.156876 26349 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 09:03:57.157078 14580 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250111 09:03:57.158802 26349 catalog_manager.cc:1348] Generated new cluster ID: e120a73205154c6aae02725d1c268776
I20250111 09:03:57.158845 26349 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 09:03:57.174558 26349 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 09:03:57.175021 26349 catalog_manager.cc:1505] Loading token signing keys...
I20250111 09:03:57.181777 26349 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866: Generated new TSK 0
I20250111 09:03:57.181896 26349 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 09:03:57.188834 26290 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48384:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:57.189317 14580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 09:03:57.191007 26364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:57.191637 26363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 09:03:57.192194 26366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 09:03:57.192688 14580 server_base.cc:1034] running on GCE node
I20250111 09:03:57.192842 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250111 09:03:57.192878 14580 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250111 09:03:57.192898 14580 hybrid_clock.cc:648] HybridClock initialized: now 1736586237192896 us; error 0 us; skew 500 ppm
I20250111 09:03:57.193699 14580 webserver.cc:458] Webserver started at http://127.14.61.1:41613/ using document root <none> and password file <none>
I20250111 09:03:57.193825 14580 fs_manager.cc:362] Metadata directory not provided
I20250111 09:03:57.193869 14580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 09:03:57.193941 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 09:03:57.194233 14580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestSeparateBeginCommitTestWorkloads.1736586070622223-14580-0/minicluster-data/ts-0-root/instance:
uuid: "cb43063a9834473cb0e75c74c45f6647"
format_stamp: "Formatted at 2025-01-11 09:03:57 on dist-test-slave-nvbt"
I20250111 09:03:57.195452 14580 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250111 09:03:57.196225 26371 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:57.196410 14580 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250111 09:03:57.196467 14580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestSeparateBeginCommitTestWorkloads.1736586070622223-14580-0/minicluster-data/ts-0-root
uuid: "cb43063a9834473cb0e75c74c45f6647"
format_stamp: "Formatted at 2025-01-11 09:03:57 on dist-test-slave-nvbt"
I20250111 09:03:57.196523 14580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestSeparateBeginCommitTestWorkloads.1736586070622223-14580-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestSeparateBeginCommitTestWorkloads.1736586070622223-14580-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFKQovg/test-tmp/txn_write_ops-itest.0.TxnOpDispatcherITest.TestSeparateBeginCommitTestWorkloads.1736586070622223-14580-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 09:03:57.202809 14580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 09:03:57.203086 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 09:03:57.203752 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 09:03:57.203830 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:57.203992 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 09:03:57.204038 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 09:03:57.208920 14580 rpc_server.cc:307] RPC server started. Bound to: 127.14.61.1:43935
I20250111 09:03:57.208937 26441 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.61.1:43935 every 8 connection(s)
I20250111 09:03:57.209753 26442 heartbeater.cc:346] Connected to a master server at 127.14.61.62:42775
I20250111 09:03:57.209832 26442 heartbeater.cc:463] Registering TS with master...
I20250111 09:03:57.209996 26442 heartbeater.cc:510] Master 127.14.61.62:42775 requested a full tablet report, sending...
I20250111 09:03:57.210500 26290 ts_manager.cc:194] Registered new tserver with Master: cb43063a9834473cb0e75c74c45f6647 (127.14.61.1:43935)
I20250111 09:03:57.211265 26290 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48400
I20250111 09:03:57.211495 14580 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002216229s
I20250111 09:03:57.212051 14580 test_util.cc:274] Using random seed: 2110420370
I20250111 09:03:57.217586 26290 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48410:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250111 09:03:57.225528 26400 tablet_service.cc:1467] Processing CreateTablet for tablet bff594df6b0948ed9b73936d97267875 (DEFAULT_TABLE table=test-workload [id=13d2bb85a449413f921f7cba1430143c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:57.225587 26399 tablet_service.cc:1467] Processing CreateTablet for tablet b718fb2b2a4c489397db0fb8fa0b0526 (DEFAULT_TABLE table=test-workload [id=13d2bb85a449413f921f7cba1430143c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:57.225528 26401 tablet_service.cc:1467] Processing CreateTablet for tablet e6bbc23021a842d9863b480e53bacf9e (DEFAULT_TABLE table=test-workload [id=13d2bb85a449413f921f7cba1430143c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:57.225848 26400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bff594df6b0948ed9b73936d97267875. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:57.226015 26401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6bbc23021a842d9863b480e53bacf9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:57.226156 26399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b718fb2b2a4c489397db0fb8fa0b0526. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:57.228061 26452 tablet_bootstrap.cc:492] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647: Bootstrap starting.
I20250111 09:03:57.228902 26452 tablet_bootstrap.cc:654] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:57.229734 26452 tablet_bootstrap.cc:492] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647: No bootstrap required, opened a new log
I20250111 09:03:57.229799 26452 ts_tablet_manager.cc:1397] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:57.230108 26452 raft_consensus.cc:357] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:57.230178 26452 raft_consensus.cc:383] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:57.230211 26452 raft_consensus.cc:738] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb43063a9834473cb0e75c74c45f6647, State: Initialized, Role: FOLLOWER
I20250111 09:03:57.230334 26452 consensus_queue.cc:260] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:57.230393 26452 raft_consensus.cc:397] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:57.230427 26452 raft_consensus.cc:491] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:57.230469 26452 raft_consensus.cc:3054] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:57.231036 26452 raft_consensus.cc:513] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:57.231140 26452 leader_election.cc:304] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb43063a9834473cb0e75c74c45f6647; no voters:
I20250111 09:03:57.231338 26452 leader_election.cc:290] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:57.231432 26454 raft_consensus.cc:2798] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:57.231631 26452 ts_tablet_manager.cc:1428] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:57.231633 26454 raft_consensus.cc:695] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647 [term 1 LEADER]: Becoming Leader. State: Replica: cb43063a9834473cb0e75c74c45f6647, State: Running, Role: LEADER
I20250111 09:03:57.231846 26454 consensus_queue.cc:237] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:57.231698 26442 heartbeater.cc:502] Master 127.14.61.62:42775 was elected leader, sending a full tablet report...
I20250111 09:03:57.232517 26452 tablet_bootstrap.cc:492] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647: Bootstrap starting.
I20250111 09:03:57.232992 26290 catalog_manager.cc:5526] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb43063a9834473cb0e75c74c45f6647 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "cb43063a9834473cb0e75c74c45f6647" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:57.233237 26452 tablet_bootstrap.cc:654] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:57.234114 26452 tablet_bootstrap.cc:492] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647: No bootstrap required, opened a new log
I20250111 09:03:57.234181 26452 ts_tablet_manager.cc:1397] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:57.234520 26452 raft_consensus.cc:357] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:57.234594 26452 raft_consensus.cc:383] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:57.234629 26452 raft_consensus.cc:738] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb43063a9834473cb0e75c74c45f6647, State: Initialized, Role: FOLLOWER
I20250111 09:03:57.234728 26452 consensus_queue.cc:260] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:57.234823 26452 raft_consensus.cc:397] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:57.234867 26452 raft_consensus.cc:491] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:57.234922 26452 raft_consensus.cc:3054] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:57.235638 26452 raft_consensus.cc:513] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:57.235733 26452 leader_election.cc:304] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb43063a9834473cb0e75c74c45f6647; no voters:
I20250111 09:03:57.235831 26452 leader_election.cc:290] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:57.235874 26454 raft_consensus.cc:2798] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:57.235968 26454 raft_consensus.cc:695] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647 [term 1 LEADER]: Becoming Leader. State: Replica: cb43063a9834473cb0e75c74c45f6647, State: Running, Role: LEADER
I20250111 09:03:57.236027 26452 ts_tablet_manager.cc:1428] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:57.236086 26454 consensus_queue.cc:237] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:57.236171 26452 tablet_bootstrap.cc:492] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647: Bootstrap starting.
I20250111 09:03:57.237107 26452 tablet_bootstrap.cc:654] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:57.237178 26289 catalog_manager.cc:5526] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb43063a9834473cb0e75c74c45f6647 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "cb43063a9834473cb0e75c74c45f6647" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:57.238106 26452 tablet_bootstrap.cc:492] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647: No bootstrap required, opened a new log
I20250111 09:03:57.238164 26452 ts_tablet_manager.cc:1397] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250111 09:03:57.238478 26452 raft_consensus.cc:357] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:57.238559 26452 raft_consensus.cc:383] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:57.238588 26452 raft_consensus.cc:738] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb43063a9834473cb0e75c74c45f6647, State: Initialized, Role: FOLLOWER
I20250111 09:03:57.238675 26452 consensus_queue.cc:260] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:57.238730 26452 raft_consensus.cc:397] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:57.238771 26452 raft_consensus.cc:491] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:57.238817 26452 raft_consensus.cc:3054] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:57.239665 26452 raft_consensus.cc:513] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:57.239809 26452 leader_election.cc:304] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb43063a9834473cb0e75c74c45f6647; no voters:
I20250111 09:03:57.239945 26452 leader_election.cc:290] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:57.239987 26454 raft_consensus.cc:2798] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:57.240069 26454 raft_consensus.cc:695] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647 [term 1 LEADER]: Becoming Leader. State: Replica: cb43063a9834473cb0e75c74c45f6647, State: Running, Role: LEADER
I20250111 09:03:57.240187 26454 consensus_queue.cc:237] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:57.240450 26452 ts_tablet_manager.cc:1428] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250111 09:03:57.241763 26289 catalog_manager.cc:5526] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb43063a9834473cb0e75c74c45f6647 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "cb43063a9834473cb0e75c74c45f6647" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } health_report { overall_health: HEALTHY } } }
W20250111 09:03:57.247550 26447 txn_manager_proxy_rpc.cc:150] re-attempting BeginTransaction request to one of TxnManagers
I20250111 09:03:58.196588 26289 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48420:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20250111 09:03:58.201071 26399 tablet_service.cc:1467] Processing CreateTablet for tablet 4e9646b61cf449508cb9d089fa8f2623 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d9e28757de9c4376b53b45a47f400388]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20250111 09:03:58.201324 26399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e9646b61cf449508cb9d089fa8f2623. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:58.203066 26472 tablet_bootstrap.cc:492] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647: Bootstrap starting.
I20250111 09:03:58.203792 26472 tablet_bootstrap.cc:654] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:58.204618 26472 tablet_bootstrap.cc:492] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647: No bootstrap required, opened a new log
I20250111 09:03:58.204694 26472 ts_tablet_manager.cc:1397] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:58.205024 26472 raft_consensus.cc:357] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:58.205116 26472 raft_consensus.cc:383] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:58.205171 26472 raft_consensus.cc:738] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb43063a9834473cb0e75c74c45f6647, State: Initialized, Role: FOLLOWER
I20250111 09:03:58.205298 26472 consensus_queue.cc:260] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:58.205368 26472 raft_consensus.cc:397] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:58.205411 26472 raft_consensus.cc:491] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:58.205459 26472 raft_consensus.cc:3054] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:58.206120 26472 raft_consensus.cc:513] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:58.206238 26472 leader_election.cc:304] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb43063a9834473cb0e75c74c45f6647; no voters:
I20250111 09:03:58.206452 26472 leader_election.cc:290] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:58.206533 26474 raft_consensus.cc:2798] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:58.206671 26474 raft_consensus.cc:695] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647 [term 1 LEADER]: Becoming Leader. State: Replica: cb43063a9834473cb0e75c74c45f6647, State: Running, Role: LEADER
I20250111 09:03:58.206799 26472 ts_tablet_manager.cc:1428] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:58.207052 26474 consensus_queue.cc:237] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:58.207401 26476 tablet_replica.cc:440] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb43063a9834473cb0e75c74c45f6647" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } } }
I20250111 09:03:58.207491 26475 tablet_replica.cc:440] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647: TxnStatusTablet state changed. Reason: New leader cb43063a9834473cb0e75c74c45f6647. Latest consensus state: current_term: 1 leader_uuid: "cb43063a9834473cb0e75c74c45f6647" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } } }
I20250111 09:03:58.207710 26475 tablet_replica.cc:443] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:58.207598 26476 tablet_replica.cc:443] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647: This TxnStatusTablet replica's current role is: LEADER
I20250111 09:03:58.208251 26478 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20250111 09:03:58.208333 26478 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20250111 09:03:58.208351 26289 catalog_manager.cc:5526] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb43063a9834473cb0e75c74c45f6647 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "cb43063a9834473cb0e75c74c45f6647" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:58.977360 14580 test_util.cc:274] Using random seed: 2112185673
I20250111 09:03:58.983851 26281 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48426:
name: "default.second_table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250111 09:03:58.988847 26400 tablet_service.cc:1467] Processing CreateTablet for tablet ad9a3055e3974cad81ae1568181beb66 (DEFAULT_TABLE table=default.second_table [id=1cc169ecf5564bf4a1e0098d4b895da2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:58.988986 26401 tablet_service.cc:1467] Processing CreateTablet for tablet 69273d3ba5714f21b43e8f8f331a7b12 (DEFAULT_TABLE table=default.second_table [id=1cc169ecf5564bf4a1e0098d4b895da2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:58.989135 26400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad9a3055e3974cad81ae1568181beb66. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:58.989457 26401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69273d3ba5714f21b43e8f8f331a7b12. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:58.989567 26399 tablet_service.cc:1467] Processing CreateTablet for tablet fb78fe9bb0f4408cb9ebf86a9562fef0 (DEFAULT_TABLE table=default.second_table [id=1cc169ecf5564bf4a1e0098d4b895da2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250111 09:03:58.989758 26399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb78fe9bb0f4408cb9ebf86a9562fef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 09:03:58.990815 26515 tablet_bootstrap.cc:492] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647: Bootstrap starting.
I20250111 09:03:58.992015 26515 tablet_bootstrap.cc:654] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:58.992871 26515 tablet_bootstrap.cc:492] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647: No bootstrap required, opened a new log
I20250111 09:03:58.992945 26515 ts_tablet_manager.cc:1397] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:58.993302 26515 raft_consensus.cc:357] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:58.993376 26515 raft_consensus.cc:383] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:58.993408 26515 raft_consensus.cc:738] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb43063a9834473cb0e75c74c45f6647, State: Initialized, Role: FOLLOWER
I20250111 09:03:58.993530 26515 consensus_queue.cc:260] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:58.993599 26515 raft_consensus.cc:397] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:58.993640 26515 raft_consensus.cc:491] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:58.993685 26515 raft_consensus.cc:3054] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:58.994458 26515 raft_consensus.cc:513] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:58.994576 26515 leader_election.cc:304] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb43063a9834473cb0e75c74c45f6647; no voters:
I20250111 09:03:58.994659 26515 leader_election.cc:290] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:58.994730 26502 raft_consensus.cc:2798] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:58.994824 26502 raft_consensus.cc:695] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647 [term 1 LEADER]: Becoming Leader. State: Replica: cb43063a9834473cb0e75c74c45f6647, State: Running, Role: LEADER
I20250111 09:03:58.994892 26515 ts_tablet_manager.cc:1428] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:58.994956 26502 consensus_queue.cc:237] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:58.995048 26515 tablet_bootstrap.cc:492] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647: Bootstrap starting.
I20250111 09:03:58.995956 26515 tablet_bootstrap.cc:654] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:58.996271 26281 catalog_manager.cc:5526] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb43063a9834473cb0e75c74c45f6647 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "cb43063a9834473cb0e75c74c45f6647" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:58.996834 26515 tablet_bootstrap.cc:492] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647: No bootstrap required, opened a new log
I20250111 09:03:58.996906 26515 ts_tablet_manager.cc:1397] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:58.997404 26515 raft_consensus.cc:357] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:58.997504 26515 raft_consensus.cc:383] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:58.997552 26515 raft_consensus.cc:738] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb43063a9834473cb0e75c74c45f6647, State: Initialized, Role: FOLLOWER
I20250111 09:03:58.997665 26515 consensus_queue.cc:260] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:58.997757 26515 raft_consensus.cc:397] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:58.997855 26515 raft_consensus.cc:491] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:58.997912 26515 raft_consensus.cc:3054] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:58.998683 26515 raft_consensus.cc:513] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:58.998796 26515 leader_election.cc:304] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb43063a9834473cb0e75c74c45f6647; no voters:
I20250111 09:03:58.998930 26515 leader_election.cc:290] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:58.998996 26502 raft_consensus.cc:2798] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:58.999096 26502 raft_consensus.cc:695] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647 [term 1 LEADER]: Becoming Leader. State: Replica: cb43063a9834473cb0e75c74c45f6647, State: Running, Role: LEADER
I20250111 09:03:58.999193 26515 ts_tablet_manager.cc:1428] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250111 09:03:58.999210 26502 consensus_queue.cc:237] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:58.999354 26515 tablet_bootstrap.cc:492] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647: Bootstrap starting.
I20250111 09:03:59.000298 26515 tablet_bootstrap.cc:654] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647: Neither blocks nor log segments found. Creating new log.
I20250111 09:03:59.000380 26281 catalog_manager.cc:5526] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb43063a9834473cb0e75c74c45f6647 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "cb43063a9834473cb0e75c74c45f6647" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:59.001330 26515 tablet_bootstrap.cc:492] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647: No bootstrap required, opened a new log
I20250111 09:03:59.001403 26515 ts_tablet_manager.cc:1397] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250111 09:03:59.001746 26515 raft_consensus.cc:357] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:59.001833 26515 raft_consensus.cc:383] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 09:03:59.001868 26515 raft_consensus.cc:738] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb43063a9834473cb0e75c74c45f6647, State: Initialized, Role: FOLLOWER
I20250111 09:03:59.001974 26515 consensus_queue.cc:260] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:59.002048 26515 raft_consensus.cc:397] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 09:03:59.002086 26515 raft_consensus.cc:491] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 09:03:59.002131 26515 raft_consensus.cc:3054] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647 [term 0 FOLLOWER]: Advancing to term 1
I20250111 09:03:59.002697 26515 raft_consensus.cc:513] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:59.002794 26515 leader_election.cc:304] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb43063a9834473cb0e75c74c45f6647; no voters:
I20250111 09:03:59.002882 26515 leader_election.cc:290] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 09:03:59.002964 26484 raft_consensus.cc:2798] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 09:03:59.003136 26515 ts_tablet_manager.cc:1428] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20250111 09:03:59.003156 26484 raft_consensus.cc:695] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647 [term 1 LEADER]: Becoming Leader. State: Replica: cb43063a9834473cb0e75c74c45f6647, State: Running, Role: LEADER
I20250111 09:03:59.003283 26484 consensus_queue.cc:237] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } }
I20250111 09:03:59.004333 26281 catalog_manager.cc:5526] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb43063a9834473cb0e75c74c45f6647 (127.14.61.1). New cstate: current_term: 1 leader_uuid: "cb43063a9834473cb0e75c74c45f6647" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb43063a9834473cb0e75c74c45f6647" member_type: VOTER last_known_addr { host: "127.14.61.1" port: 43935 } health_report { overall_health: HEALTHY } } }
I20250111 09:03:59.125634 14580 tablet_server.cc:178] TabletServer@127.14.61.1:0 shutting down...
I20250111 09:03:59.129806 14580 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250111 09:03:59.130178 14580 tablet_replica.cc:331] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647: stopping tablet replica
I20250111 09:03:59.130306 14580 raft_consensus.cc:2238] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:59.130406 14580 raft_consensus.cc:2267] T fb78fe9bb0f4408cb9ebf86a9562fef0 P cb43063a9834473cb0e75c74c45f6647 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:59.130895 14580 tablet_replica.cc:331] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647: stopping tablet replica
I20250111 09:03:59.130978 14580 raft_consensus.cc:2238] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:59.131068 14580 raft_consensus.cc:2267] T bff594df6b0948ed9b73936d97267875 P cb43063a9834473cb0e75c74c45f6647 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:59.131496 14580 tablet_replica.cc:331] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647: stopping tablet replica
I20250111 09:03:59.131572 14580 raft_consensus.cc:2238] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:59.131667 14580 raft_consensus.cc:2267] T b718fb2b2a4c489397db0fb8fa0b0526 P cb43063a9834473cb0e75c74c45f6647 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:59.132112 14580 tablet_replica.cc:331] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647: stopping tablet replica
I20250111 09:03:59.132205 14580 raft_consensus.cc:2238] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:59.132297 14580 raft_consensus.cc:2267] T 4e9646b61cf449508cb9d089fa8f2623 P cb43063a9834473cb0e75c74c45f6647 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:59.132687 14580 tablet_replica.cc:331] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647: stopping tablet replica
I20250111 09:03:59.132762 14580 raft_consensus.cc:2238] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:59.132843 14580 raft_consensus.cc:2267] T ad9a3055e3974cad81ae1568181beb66 P cb43063a9834473cb0e75c74c45f6647 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:59.133270 14580 tablet_replica.cc:331] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647: stopping tablet replica
I20250111 09:03:59.133347 14580 raft_consensus.cc:2238] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:59.133430 14580 raft_consensus.cc:2267] T e6bbc23021a842d9863b480e53bacf9e P cb43063a9834473cb0e75c74c45f6647 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:59.133787 14580 tablet_replica.cc:331] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647: stopping tablet replica
I20250111 09:03:59.133850 14580 raft_consensus.cc:2238] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:59.133936 14580 raft_consensus.cc:2267] T 69273d3ba5714f21b43e8f8f331a7b12 P cb43063a9834473cb0e75c74c45f6647 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:59.147377 14580 tablet_server.cc:195] TabletServer@127.14.61.1:0 shutdown complete.
I20250111 09:03:59.149863 14580 master.cc:537] Master@127.14.61.62:42775 shutting down...
I20250111 09:03:59.153450 14580 raft_consensus.cc:2238] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866 [term 1 LEADER]: Raft consensus shutting down.
I20250111 09:03:59.153551 14580 raft_consensus.cc:2267] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250111 09:03:59.153609 14580 tablet_replica.cc:331] T 00000000000000000000000000000000 P 3137585be172470ea2a6689a1be1a866: stopping tablet replica
I20250111 09:03:59.166453 14580 master.cc:559] Master@127.14.61.62:42775 shutdown complete.
[ OK ] TxnOpDispatcherITest.TestSeparateBeginCommitTestWorkloads (2040 ms)
[----------] 18 tests from TxnOpDispatcherITest (79349 ms total)
[----------] Global test environment tear-down
[==========] 26 tests from 2 test suites ran. (168540 ms total)
[ PASSED ] 25 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] TxnOpDispatcherITest.TxnWriteWhileReplicaDeleted
1 FAILED TEST
YOU HAVE 1 DISABLED TEST
I20250111 09:03:59.172886 14580 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 3 messages since previous log ~10 seconds ago
I20250111 09:03:59.172935 14580 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tablet/tablet.cc:2367: suppressed but not reported on 9 messages since previous log ~23 seconds ago
I20250111 09:03:59.172962 14580 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/transactions/txn_status_manager.cc:765: suppressed but not reported on 100 messages since previous log ~12 seconds ago
I20250111 09:03:59.172988 14580 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tablet/tablet_replica.cc:1307: suppressed but not reported on 2 messages since previous log ~3 seconds ago
I20250111 09:03:59.173089 14580 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/master/master.cc:440: suppressed but not reported on 3 messages since previous log ~7 seconds ago
I20250111 09:03:59.173156 14580 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/client/txn_manager_proxy_rpc.cc:150: suppressed but not reported on 6 messages since previous log ~1 seconds ago